In my defense:
1) There's definitely an element of "if the user can’t find it, the feature doesn't exist" with downvoting here.
2) In practical terms, downvotes are so extraordinarily rare on HN that they almost don’t exist.
Anyway, I'm encouraged that downvotes exist on HN, but the threshold could be considered so absurdly high that they're nerfed into oblivion.
It's not because you don't see them directly a lot that they don't have an impact. A comment that ends at +3 with 7 upvotes and 3 downvotes would have ended at +7 (or more because of the "momentum effect") if there weren't downvotes.