As an outsider sometimes you are not aware of cultural norms, but I always felt the margin for error for what Americans considered "acceptable" was too narrow. There was no benefit of doubt, and sometimes an explicit dislike for someone who did not immediately fit in.
It's worth noting that in the US, that's a stereotype of the west coast and California in particular. The contrast with NYC is particularly frequently cited. New Yorkers have a reputation for being very direct and coming across as rude but will actually be very nice and helpful once you get past that. New Yorkers will call you names that would make a sailor blush, and then immediately rush to help a woman carry her stroller up the subway steps.