I never understood people acting like downvoting is some huge crime or being downvoted is some great slight. It's a little arrow on a web site. It makes a number in a database go up or down by 1. I don't care how many points my comments score, and neither should anyone else. Speak your mind, or don't, and move on with life.
If you vote according to your own views on a topic, then you're saying that you want everyone to see only views that agree with yours, and you want contrasting views to be hidden from everyone. This must be true because voting affects the presentation of comments on the site; it is not at all just an inert database column.
I would like to see well-expressed opinions regardless of how echoey or reprehensible they might be. I visit HN in the hopes of having my mind changed, and those kinds of comments are the very comments that might change my mind.
That's why I especially try to vote for any comment I reply to (because it was literally thought-provoking); comments that restated another jumbled comment well (they improve site readability); and comments that explain extreme viewpoints in rational if not reasonable terms (they promote further logical or at least empathetic discussion). None of these have anything to do with my personal opinion on the subject.
You're going to downvote my comment because I think voting serves to curate the site to make it a better place to visit, and you think voting is a popularity contest for your own world view. I want your viewpoint to be seen and discussed. You want mine to be hidden.