I'm pretty sure white atheists in America feel equally among "my people" as Christians do (unless they're in a church or other highly religious surroundings).
It's also an odd comparison as religion is a choice, where race or ethnicity is not.
As for ethnicity, while again technically it is not a "choice", I'm not really sure it can be clearly defined. A friend was mine was born in Italy to British parents, his wife was Bosnian and two of their children were born in the US -- what is their ethnicity?
It's not clear to me what aspects of religion would be obvious candidates for being described as a 'choice'. Personally, the fundamental aspects seem to operate as primal psychological forces, and overt action in contravention to them rapidly erodes things like confidence and self-esteem, while increasing things like anxiety. Every once in a while, people will definitely work against those pressures, but I would expect that behaviour to be unsustainable in the long term for the vast majority.
I'll also ask that atheists stop referring to religion as a choice. Atheists have as much of a say in whether God exists as anybody else. It would be patronizing to tell firm atheists that they chose atheism when clearly in the atheists mind, they are just reacting to reality. Likewise a theist doesn't have a say in whether an almighty creator exists and has opinions or not.
To be clear: I'm not trying to compare being Jewish to being Black. I'm saying that I can draw empathy from the Jewish part of my identity toward other people who have been mistreated because of part of their identity.
Also, even if it were a choice, I would no more give it up than anyone else should have to give up their race or ethnicity or any other part of their identity just to fit in.