If any non-zero subset of reasonable people are so offended by a behavior that they'd leave the industry because of it, we have to cut it out.
So don't ask "would this bother me?" Ask "would it bother someone?" And since you can't predict this from inside your head, you have to rely on firsthand accounts of people being bothered. This seems like a good overview of such accounts.
One easy rule is that if someone says "Only talk to me about work." then the other person has to respect it. No forcing of social acceptance , no shaming the other to believe what you believe, just focus on what you were hired for. This is a standard taught to many managers to keep the company out of harassment issues , its very robotic unemotional but its clear and will allow different groups to work together as long as this rule is enforced.
Basically we should not have to worry about a toxic culture because you should not be forced into one when you work. You should just be able to work and separate yourself from your task in any emotional way.
[California dev, 8 years, have held manager position]