In one sense it's a shame when thoughtful, evidence-based discussion is discouraged for being off-topic. But I suspect that's ultimately what makes those discussions possible; they're happening between relatively small numbers of discussants, in a space that doesn't draw in people looking for political debates.
Actual abuse, trolling, etc deserves to be downvoted into unreadability.
In one sense it's a shame when thoughtful, evidence-
based discussion is discouraged for being off-topic. But
I suspect that's ultimately what makes those discussions
possible;
Thoughtful, evidence-based discussion should never be discouraged or deemed off-topic - no matter how sensitive the topic. ( People reading this, even 10-15 years on, might find the mores of this age whimsical, at best. )
The only reason one might find thoughtful, evidence-based discussion off-putting is to be on the good side of David Geffen in the hopeful attempt that he or she might be might be invited to luxuriate on his super-yacht, one day.
There's never a better reason than that to be economical with the truth.
Off-topic has nothing to do with the sensitivity of the topic or the current social mores. It has to do with not being on topic.
It is near impossible for a group to have a rational discussion about politics or religion. So it is better to just avoid the topic on a forum that values, pleasantry and support of one another as core values.
I'm saying there's benefit to moderating off topic discussions regardless of the nature of said content.
How often do you hear "I didn't know that, and it's a good point" in political discussion? When hypothetically, there's no reason you should hear it less frequently than in scientific discussions.
One of the great casualties of modern political debate is that citizens mimic professional politicians, in that the sole mode of discourse is argumentative.
When in reality, if I'm faffing about on HN I would much rather learn something than "win."
It's not like dang steps in at the end of every debate to award the winner a gold trophy.
I think most people would accept that thoughtful, evidence-based discussion should be discouraged while sitting in the front row at an opera, or in the midst of a professor's lecture.
What I'm describing is similarly an issue of logistics, not content. I'm not making a claim about sensitive topics, and I'm certainly not proposing dishonesty or the suppression of uncomfortable truths. The problem with off-topic content is simply that: it's off topic, and on a forum thread or the top of an HN post it makes on-topic conversation more difficult to conduct. Forked-discussion settings like Tumblr and Twitter are closer to a conference than a lecture, and can sustain popular off-topic discussion with less derailment.
The relevance of politics and sensitive topics is only in my second point, that places like HN which center on non-political topics can create particularly good discussions. I largely agree with you, I'm endorsing the fact that HN doesn't ban politics or sensitive topics; the rules of avoiding flamebait, grandstanding, and excessive derailment help to prevent pointless yelling while preserving good political discussion around the margins.
(As an aside which risks being off-topic: why David Geffen? I've never seen someone use him as their go-to example of sucking up to a billionaire.)