If this were my product then I'd try to gather a corpus of bad comments, selected outside of the vote system (because the problem is that voting might be broken). While I was at it, I'd also find out the good comments, because promoting good comments might be just as good and easier than getting rid of the bad comments. After that, I'd try to figure out what counts as a good vote or a bad vote, because the problem probably doesn't really lay with the comments themselves, but rather how people vote against them. Bad comments aren't really a problem if the vote system does a good job of spotting them.
Then I'd take a careful look at comments and votes:
- Is the distribution of good comments / bad comments even throughout the set of commentators, or are there users who are dependably good or dependably bad? If it is a lumpy distribution then you can use that. I'm guessing that everyone makes dumb comments, and there is something with the system that inflates the scores of bad comments compared to good comments as more people can vote. But I'm also guessing that only so many people are capable of leaving good comments too. Get the data and find out for sure.
- Do the vote scores that these comments get a reflection of the quality? If the votes are, then maybe the system isn't as broken as you think. If they aren't, then you've got a lead on the problem - you can look at the bad comments that get lots of upvotes and try to suss out what is going on.
- Do high-karma voters do a better job of finding good / bad comments that average? If they are better, then maybe you give them more weight. If they aren't then you'd have to shelve that idea.
- Are there people good at commenting but bad at voting, and vice-versa?
- Are there people who are good at upvoting, but not at downvoting, or vice-versa?
It's all sort of tedious, but basically I'd advise leaning on the data and make decisions based off of that. I'm pretty sure that if you dig in a bit something is going to really stick out in a big way. Once you've found that, then you can build a feature / change around that.
This is very time-consuming to do, however. Writing a detailed response is costly and repetitive. Pressing "downvote" is easy. You punish the commenter, but don't tell him/her why; it's up to them to figure it out on their own, which is sometimes difficult/impossible from their perspective.
An idea:
If you downvote, you have to pick a reason why you downvoted from various options (e.g. drop-down list, checkboxes). If you pick something, your downvote is cast. If you don't do anything, the downvote isn't cast.
In the options, you can list ~10-20 reasons to downvote (too mean, off topic, etc...). Select one or more items from the list, and submit it. What was selected appears in a "voting stats" page/section for said comment. Then the user can then get the gist of why they were downvoted, perhaps with a generic message saying how they could improve in the future.
The downside is that it might required a re-direct if you downvote (might be annoying on iPad/mobile devices), and it'll also take longer to downvote. I don't think the latter will be an issue since downvoting is rare, and I'd imagine people would be willing to fill out a quick web-form if they really think something deserves negative karma.