Sometimes (often?) people with a lot of money may not believe in speech but in suppressing speech. However, money should not allow for suppressing speech, for example by buying a giant megaphone and speaking over people.
By your logic paying people $500 to heckle at your political opponents rally is fine. It may be legally okay, but it is a moral hazard, and for a better society we should try to better distinguish between “free” speech and “bought and paid for” speech.