At this point I'm actively looking for a replacement community that focuses much more on ML and Linux, and not on the social issues associated with machine learning, or social justice warriors messing up tech firms, or people who seem to get their "science" from Fox News.
If you word things in a confrontationally, overly partisan, or ignorant way, you'll get downvoted, even by people who technically agree with you. So, go out of your way to write more neutrally to reduce downvotes on tone.
If you propose an unpopular idea, or criticize one of HN's Holy Cows, you'll get a bunch of downvotes from people who disagree with you. Occasionally, I don't comment at all if I think my opinion is out of the mainstream and not justifiable.
Some folks just downvote people who state the truth because they don't like negative naysayers. Make it clear, if you're stating How the World Is, that you don't necessarily agree with it, but it's a structural problem.
Other folks don't like simple solutionism so avoid saying "It's easy. We can solve world hunger by <blahblahblah> idea" which will never work, because food production isn't the reason people are starving.
Another thing that can get you downvotes is invoking Expert Privilege. For example, HN will downvote you if you don't provide some sort of pseudo-rational sounding argument and instead just say you're an expert. OK, fine, ignore the fact that my PhD taught me how to read science papers and press releases.
Finally, and this is the most interesting thing, votes come in waves. I'll often get -2 on a comment right after I post and then it will trend upward for a whole day. I suppose my comments age well.
Ultimately, maxxing my karma is generally correlated roughly with making good contributions to the site, and I've calibrated well enough to interpret downvotes.