Yes, as a man, sometimes you do have to be careful about how you give feedback to female founders, coworkers, or employees. It's a skill to be developed just like 1,000 other little skills you need to develop be reasonably successful in life.
This same "but men might get in trouble for an innocent mistake" argument was used when women first entered the workforce to decry the unreasonable burden having to work with women placed on men. You still see variations on it today.
Yes, Twitter mobs are a thing and the people who participate in them are horrible. But if you are having trouble giving feedback to women, that is not women's fault. It is simply a skill you need to develop.
I've been in plenty of panel discussions where I have offered advice to female founders publicly. I was in a semi-open mentoring session just a few days ago when I did so. In fact, I've been called out in the pas for not being sensitive to women's perspectives. (No twitter mobs, thankfully)
Giving good feedback is a skill. Giving good feedback to people of different races, genders, ages, nationalities, etc is an important subset of that skill.
Regardless, I do believe you aren't properly weighing the cost/benefit or risk/reward properly in public v private situations. One can have all the skills in the world, it isn't going to get the risk to zero. And if the reward is little, it doesn't add up for many folks.
I'm not a public person so I'm speculating.