Really bad comments are not the root of the problem. Simply having large number of mediocre comments crowds out and discourages thoughtful discussion from starting at all.
I'd say:
* create some real cost to making comments
* make bad comments disappear/not display at all with time
* make things less democratic -- to encourage good behavior identify users who have this behavior and make this behavior more prominent programmaticly
A karma penalty is a very soft slap on the wrist, at best.
It won't stop trolls, or assholes, or anyone with an agenda.
Subtracting a fixed amount of karma also gets less effective the more karma one has accumulated. So people with a lot of karma will be able to get away with more than people with less karma.
This has an upside, in that it allows more valued members of the community to express themselves more freely. The downside is that they can act like assholes without much repercussion. If the community winds up rewarding them for acting like assholes (by, say, upvoting their assholish comment) then that's even worse.
Of course, any community that not only tolerates but encourages assholes is not a community I want to be part of, and I've left a number of communities over that sort of behavior.
But there are other solutions short of "love it or leave it". I describe one such alternative in another comment in this thread: