It's not only X% chance of being accused of sexism, but a very high chance to lose one's career - and possibly become un-hireable by any large company or to any position of responsibility at least for a while. It's not just that someone says "you're sexist" and you say "no, I'm not!" - it goes way beyond accusation, and accusation itself almost universally is considered as good as definite proof. This is a serious risk, and it's totally understandable that people want to avoid it.