Does it seem to you that shard and sherd have something shared? Their shape, of course, but more: they name shreds of things that don’t shred – they smash or shatter as they are dashed against something hard.
In fact, according to Oxford, the two are the same old Germanic word, just different spellings. But is that still true? Have they not by this time diverged somewhat, as person and parson or perilous and parlous or vermin and varmint or any of a fair few other pairs that have grown apart to some degree? Would you use them in exactly the same way?
They have a difference of sound, to be sure. Shard is wider open but also sharper, and indeed has echoes of sharp and shatter and hard. When you think of shards, what material do you think of? I suspect broken glass comes to mind first. Glass is certainly the most common noun to go with shard.
When you think of sherds, on the other hand, can you think of broken glass? Perhaps some can; I cannot. It has to be ceramic. The most common nouns that go with sherd are pottery and ceramic, and there is also the word potsherd that shows up quite often.
So think of these two words not as the same word, not as twins that have grown apart, but as fragments of some frangible thing – a pot, perhaps, with a repeating pattern on it. One of them has most of the pattern, but is stronger on one side of it; the other one has less of the pattern and is limited to the other side of it. There will be no mending or replacement, either; the pot wasn’t insured, and now it’s in sherds.