I find it very disheartening that the negative voices are being given so much weight. Everything that's worth doing will have detractors, and when it's something really worth doing it will have vocal detractors. Back when I had comments on my blog, every article I wrote that was any good had at least one person commenting that I was a moron or some equivalent statement.
Great things arouse passion - on both sides.
Giving 10x the power to the people on the negative side just creates an environment where new ideas are discouraged, where important but difficult discourse is pushed aside, where things of true import are penalised out of the group's attention by a few detractors.
There does need to be a system for flagging and removing spam articles, but if this system can (as it plainly regularly is) be co-opted to remove articles from sight just based on not liking them much, then it is broken. The people who have flagging powers are not responsible enough to use them wisely, perhaps.
I see at least one simple solution: lift the flagging privileges so it only becomes available to a much smaller segment of the population. Perhaps making the limit 10'000 instead of 500 would do that. That would still include hundreds of people, based on a quick extrapolation from https://news.ycombinator.com/leaders ). An even better model would be to make it dynamic - perhaps the top 200 commenters...
As to flagging, I think a bigger problem is activism-upvoting, in other words upvoting so others see a story and not because it's interesting.
If you don't trust the average user with flagging powers because of how they affect the front page, why do you trust them to upvote good articles onto the front page? Clearly only select users should be able to upvote articles.