I ask because if you took that same attitude towards race, you could easily be accused of colorblind racism.
When I was young I had something similar happen. A woman went around telling people she had slept with me. We hadn't. Nobody believed my side of the story since "why would she lie?". This caused a rift with my best friend, who had a crush on her. I lost my best friend due to a lie I couldn't disprove.
To be clear, I'm not saying that I do treat all people equally, regardless of their race, gender, sexuality, etc. I grew up in an environment filled with stereotypes, and they do seep through. I can only strive to avoid having that bias affect how I treat people. If it does leak through, I try to recognize it and do better in the future.
I don't think that treating people poorly because you're worried that not doing so could hurt your career is the right thing to do. It might be pragmatic in your case, but so is taking money out of a wallet you find on the ground.
> I lost my best friend due to a lie I couldn't disprove.
You lost your best friend because your best friend didn't trust you and because someone lied. None of that is your fault. Sometimes the world is a shitty place with shitty people. That isn't a reason to add to that shittiness by refusing to treat women in the workplace as you would men.
Some people would go as far as to say this is a cornerstone of white supremacy.
Similarly, if someone's background (say gender) results in fewer mentorship opportunities because of the ol' boys club, I'm happy to put in some effort to put them in touch with people.
The point I am making is not about whether you are racist though, it’s that saying that you treat all races equally is enough to be classed as racist, regardless of your actual behavior.
I've returned every wallet I've ever found and I've never taken any money from them. Doing so poses no risk to myself. It's very easy to stand on your high horse and criticise others but after you've been in the situation of being accused of something it changes your opinion on certain things. I hope that you never have to deal with it.