Consider the case of a sentence such as the following:
I must agree with whomever wrote this.
Is that correct?
Nope. It should be whoever. The relative pronoun here (whoever) is the head of the subordinate clause, which as a whole stands as the complement of the with. Many people see the pronoun as the object of the with and assume it must be object, as in “I must agree with whom?” In this case, however, it is not who(m)ever that is the object; it is the whole subordinate clause: “whoever wrote this.” A clause needs a subject, and when one word is serving as both relative pronoun and subject, it is the subject and takes the subject form.
It would be different if it were a sentence such as “The error is that of him who wrote it,” as in that case him is just the object; “who wrote it” is a whole clause modifying him, and who serves as the subject of that clause. (See “Let her who is without error…” for an extended discussion of this.)
Contrast it also with a sentence such as the following:
Comments on YouTube are generally written by adolescents, most of whom are grossly immature and barely coherent.
Should that be “of who” because there is an are after it? No, because most is the subject of the clause: “most … are grossly immature.” The “of whom” modifies most (your hint is the of) while at the same time referring back to adolescents.