Suggestion: It seems reasonable that some comments will be accurately valued at 0, or -1... maybe even -5. But from time to time, comments get forced down to -10, -15, -20... and that just seems to be taking it too far.
Perhaps have either a hard lower limit, beyond which it just doesn't make sense to go, or, have a soft lower limit, beyond which moderator must have some minimum karma score to be able to downvote further.