In fact, there's almost a reverse incentive; if it clashes with the rest of the city's aesthetic, you're much more likely to notice it.
I don't really see how it's hurting "free speech" to restrict billboards. No one is suggesting we regulate the speech, no one is telling these companies what to say, we just don't want big ugly billboards blasting in our face and making our cities look terrible.
You're of course free to disagree with this, but you almost certainly draw the line somewhere. If I went and dumped a bunch of trash and feces into the middle of the street every day, you probably wouldn't be outraged when I eventually get a ticket, and I doubt that making a "free speech" argument would get me out of that fine, even if I explained the artistic merit of me doing that.