We already have a term for it: victim blaming. Just because someone is using a valid issue (sexism) doesn't mean they aren't using it as a weapon. When the victim acts in a more cautious manner that's perfectly reasonable. To call them out on that (non-imaginary) fear is victim blaming. The abuser in this case is someone with a personality disorder or some other mental health problem that means they will use any tools at their disposal to manipulate or control another person. It's not sexism, it's prudence. One would use the same prudence against men who used something to manipulate or control them. It would look different, but it would be the same thing.