If you would prefer to withhold a deserved apology to avoid being perceived as "weak", that's your prerogative. However, making a sexual stereotype and then refusing to acknowledge it is a means of perpetuating sexism. It's true that systemic change is needed to eliminate sexism and racism, but society does not change all at once: every action (including every apology) contributes to the solution.