Enough HN members are happy to feed the trolls (even if unwittingly) and encourage more without removing them.
Downvoting, flagging works even when the mods are not around. I am pretty sure that a comment flagged too many times is greyed out and almost invisible (and that happens without mods).
Again, if we are to believe there are only 2 full time moderators on HN for the amount of comments going every single minute, it's virtually impossible to rely on mods alone for proper discussions: simple maths.
I run a group of 20,000+ people, and if I didn't set the tone on what the group is about, regularly, the value of it would drop to nothing.
I love my community. But I absolutely recognize the role I play in keeping it a nice place to visit.