To the OP's question, since the aggregate effect of downvoting is suppressing the visibility of a comment, I think of a downvote as my personal "delete" button.
Thank you for making this point - you stated it perfectly. Regardless of what you think downvotes "should be used for", the fact is that they are used to "downvote", whatever that means, and that alone is valuable.