The consequences are a hit to your reputation (I think they call this "being canceled" nowadays). I guess you can get to a point of wealth where this doesn't really matter. But at least for 99% of people this is really harmful and takes a lot of effort to rectify.