This word happens to immediately put the song “When I’m Sixty-Four” by the Beatles in my mind: “Every summer we can rent a cottage, in the Isle of Wight, if it’s not too dear. We shall scrimp and save. Grandchildren on your knee, Vera Chuck and Dave.”
And indeed scrimp and save is one of the most common collocations in which to find this word. It is also commonly used with on plus a noun phrase – here’s a real live example: “For the most satisfying results, don’t scrimp on the olive oil or the salt.” This example also shows the other thing that shows up often with scrimp: n’t – scrimp is frequently preceded by a negative auxiliary, inevitably in contraction: don’t scrimp, shouldn’t scrimp, doesn’t scrimp, can’t scrimp.
The word is a bit of a shrimp, really, which is appropriate given that shrimp is a closely related word (and one that was used to describe small creatures and people generally for some time before being applied specifically to the cocktail-party crustacean, which has given us that entertaining oxymoron of entertaining, jumbo shrimp). Scrimp, for its part, has origins with Germanic words relating to shrivelling, wrinkling, etc. It hit English as a verb, adjective, and adverb in the 18th century; a noun version came into use in the 19th century. These days we scrimp on all but the verb. It’s as though we’ve crimped the outflow of the others.
Scrimp also avails itself of a notable phonaestheme, the scr onset. Scr often goes with words that involve roughness and/or constriction (scrabble, scrape, scree, scraggly, scrap, scratch, scruffy, scrofula, scrub, scrunch) but also shows up with many words of writing (scrawl, screed, script, scribble, scribe, scripture, scrivener, scroll – several of these trace to the same Latin root). There are some others that have different meanings but may still bear the aesthetic influence of association with the preceding lists: scream, screech, screen, scrim, scrutinize, scry, scram, screw, scrimshaw, scrod, scrotum, scrounge, scruple… and quite a few more.
The imp rime has less of a clear effect: limp, blimp, pimp, shrimp, gimp, wimp. I leave it to the reader to taste the associations, but they are not overall positive.
Serve this word in a variety of levels of text, all but the most formal and most informal, but mostly focused on practicalities.
Sesquiotica 
I have never agreed that ‘jumbo shrimp’ is a true oxymoron. It’s like saying ‘small shark’, since in this case, ‘shrimp’ refers to a species, not a size.
It might be amusing, to ‘wordies’ who know better, to ironically call it an oxymoron. But only until people who don’t know better start believing it…
True — a point I have actually occasionally made in the past, for which I have been rewarded with rolling eyes — but since “shrimp” was first of all a word for a small thing, and then after that specifically for that small thing, there is still a sense of smallness to it (as witness its persistent use for referring to small people etc.). Size is relative, of course; one may speak of a large atom. But still and all, there is an unavoidable sense of contrast in “jumbo shrimp,” wouldn’t you say?
Yes, it could be an oxymoron, it just usually isn’t.
And I totally hear you about the rolling eyes; I get them too.