I would be surprised if this had much of an impact. I've encountered a lot of people even on HN who truly seem to believe their ideology is somehow 'fact'. Those who are sufficiently self-aware in this regard are probably not abusing the downvote button to begin with.