This is an empirical science. Any current consensus rules about what's acceptable derive ultimately from listening to people who are bothered by things. For example, in the 60s women started to object in large numbers to being wolf-whistled at when they walked past. Gradually, that became unacceptable in most workplaces. Many more things have become unacceptable since, and many more will in the future.
So there's no fixed rule, just a dialectic where people who are offended by things speak up, and people who run organizations listen and ban things that offend the most people. The process is always frustratingly slow, but it seems to be mostly moving in the right direction.