As an outsider sometimes you are not aware of cultural norms, but I always felt the margin for error for what Americans considered "acceptable" was too narrow. There was no benefit of doubt, and sometimes an explicit dislike for someone who did not immediately fit in.
It’s a terrible custom in most ways but I don’t think you got a good explanation.