Do what you like. What I like to do is not downvote, because that feels too negative; I generally assume people are earnestly contributing, even if it feels troll-y.
There's also a technical side: I tend to read HN on mobile, and to vote I have to pinch-zoom unless there's only an upvote option, as with titles (I would like that option in settings for all posts).