Another thing I've considered is having specific types of flags on comments, and having them have different effects. E.g. there could be a flag for incivility, and if you got enough of those (maybe in proportion to your total number of comments) you'd actually get kicked off the site temporarily.
My perception of the flag button now is that it's the bat-signal; it means that something is so bad that it actually warrants individual admin attention.