Some cultures have known that things are not binary for a very long time:
What constitutes the gender role of "a man" or "a woman" is fluid, not well defined, and subject to change. What constitutes "femininity" and "masculinity" is also fluid, not well defined, and subject to change.
Even if sex was a binary (which it isn't, but it's not a terrible argument to say that it is close enough to one for many purposes), when it comes to gender we all exist in a multi-dimensional space with so many variations on so many themes. Insisting that gender is binary is so harmful, even to people who consider themselves as being at one or other end of that binary. It's fine that there are people who fully embody a particular Victorian-era notion of masculine and feminine (or any other one, really), but the vast majority of us are nowhere near that simple. Insisting that gender comes in only two forms, and has no fluidity to it hurts all of us.
Being "a wife to start a family" requires a person with female sex, and is typically associated with female gender. But that association is not required, and has not been so across all human cultures and all time.
FWIW, I try to use "male/males" to refer to sex, and "man/men" to refer to gender, since AFAICT, there are no terms that clearly refer to gender.
"men" and "women" should not have their meaning diluted. They already have the meaning of "male" and "female".