Maybe this is a consequence of Hacker News not having a way to downvote stories?
I only flag stories that are blatant violation of HN's guidelines: SPAM, politics, racist... Otherwise, if I don't like a story I don't do anything.
Maybe I'll start flagging stories that I don't like?
If as many people thoughtfully vouched as maliciously flagged it may be less of an issue.
I also pretty regularly vouch.