I feel like "giving into the fear of being labeled" doesn't fully capture the risk involved. For many people that labeling means the end of their career, or at the very least a lot of personal and professional embarrassment, plus a big negative mark on their record. I have a hard time looking down on anyone too hard for giving in to that fear.