I don't see many people admitting that they downvote opinions they disagree with, but I will. It's a simple way for me to assert "I disagree," even if the act will have no visible impact (unless many others do the same). It also lets me track which comments I've downvoted (via the "undown" tag) vs. those that I have upvoted.
Sometimes it's difficult to decide whether I should upvote or downvote a comment. Opinions are often nuanced, and there are many cases where I may disagree with some points but agree with others, or recognize that even though I disagree with the commenter, that person has put forward a strong or well-articulated argument.