a) If a comment is truly mean, a personal attack on another community member, delete the comment and subtract from the user all the karma that the comment gained. That is something that can only be done by someone with curator powers here, but the rest of us can be encouraged to flag such comments more, and reminded not to upvote them.
b) If the comment is dumb, make a better comment in the same thread and downvote the dumb comment, especially if the dumb comment already has significant karma accrued. Anyone who has downvoting power (and user who has made many upvoted comments) can do all of that, and anyone who can post a comment can do some of that. Again, the curators can remind users from time to time to maintain those standards.
c) All users can browse the bestcomments list
http://news.ycombinator.com/bestcomments
to search for massively upvoted comments that are still within the downvoting time limit, and downvote those that are mean or dumb. Curators can delete those comments as needed.
Example and reminders go a long way. (By the way, because I, and I suppose most users, don't read this site exhaustively, I'm not fully aware which recent comments may be the most problematic. But definitely feel free at any time to provide me or other users with advice on how to raise the quality of comments here.)
After edit: another comment from another user in this thread prompts me to ask whether all new users who sign up see the site guidelines automatically or not. That might also help a little, if it isn't already done. Posting links to the site guidelines in threads with problems might also help.
That is interesting. Take it one step further: if this is set up (or if it is already), requiring them to pass a short quiz can ensure they actually read it with comprehension.