This lexeme is meant to be vexatious.
It is in a class with a nice few others, all invented by Lewis Carroll for his poem “Jabberwocky” from Through the Looking-Glass. In that poem we learn that a sword can be vorpal, that thought can be uffish, that a wood can be tulgey, that a boy can be beamish, and also that toves can be slithy, borogoves mimsy, and Bandersnatches frumious (but can Cumberbatches be benedict?). And we learn that a foe can be manxome.
And we have precious little other than context and form to guide us on the senses of these odd words, as they are all isoglosses, deliberately so – or, anyway, were at the time they were written: confections all, dropped as hot congealing sweet goo from the sugary mind of Charles Lutwidge Dodgson (Carroll’s real-life name). For some of them, comments made elsewhere give hints; for instance, brillig (as in “’Twas brillig,” the first two words of the poem) is not an adjective of quality such as bright or brilliant or something like that but merely a reference to the time of the day: around 4 pm, the time to begin broiling things for dinner, as we learn from Humpty Dumpty just after the poem in the book. But manxome? No hints.
There are ideas, of course. Manx appears to the eyes right away, but why the Jabberwock should have anything to do with the Isle of Man is perfectly unclear; no one has found anything to say that Dodgson hated the Manx. Some suggest it could come from manly plus buxom – bearing in mind that until fairly recent times buxom could be applied to anyone or anything and meant ‘pleasing, amiable, compliant’ (from Old English buh ‘bow, submit’ and sum, the source of the same suffix we see on gladsome, noisome, fearsome, etc.). But I do not think any sense of buxom Dodgson might have used could suit a daunting foe.
The Oxford English Dictionary has its thoughts (though, amusingly, Wiktionary does not, nor does Urban Dictionary). It suggests that it traces to manky, which means ‘gross’ or ‘crappy’ or ‘yucky’. Manky might trace to an old word mank meaning ‘maimed’ or ‘mutilated’, or it might trace to French manqué, ‘missing, wanting, defective, unsatisfactory’. Or the mank could just be echoing rank and dank and stank. In any case, the mank crosses swords with some to give us manxome, and in any case, the hero of the poem would be going off into battle against a shoddy, gross, disgusting, generally unpleasant foe, probably a wonky and jabbering one at that.
What we can say we know is that manxome describes some unpleasant characteristic of a foe. And it intentionally defies clear understanding (that’s the point of the poem). It is vexatious and ‘vexatious’, it seems, most clearly defining itself by its own lack of clarity. That is to say: it is not its definition but the nature of its indefinition that defines it.
What, you think that’s impossible? If you can’t believe impossible things, to quote Through the Looking-Glass again, “I daresay you haven’t had much practice. …Why, sometimes I’ve believed as many as six impossible things before breakfast.” Some of them rather manxome, I might add.