I think it's true and fair to say that caring more about how people are perceiving you results in drawbacks, and the world we live in where people do watch how their actions effect others isn't a perfect, problem-free world.
It makes intuitive sense to me that sometimes, when we work to raise people up, we do so at some cost to the people who are already at the top. This could be an example of that, I think.
My point was if there are many women who are benefitting from men being more thoughtful and careful, then it could be worth it to pay the cost of some women not benefitting, especially if they're approximately at the top already.
I don't think demanding a change be strictly positive for everyone is a fair evaluation of the change.