But OPs point is broader: if you allow the bad people to just appropriate the symbol as their own, they're going to gradually take over everything. Never mind swastikas; we're at the point where making an okay sign can be misconstrued as a white nationalist gesture, and people self-censor themselves accordingly.
There's also the reverse problem here, where, if you tie such things so strongly to symbols in popular opinion, then loud condemnation of such symbols is used to "prove" that one is not a bad person. For a major ongoing example of this look at Russia with its cult of "we defeated the Nazis therefore we're definitely the good guys".
At the end of the day, it's really just a lazy shortcut. The bad people are bad because of their ideas and actions, not because of their symbols. If we always look at the ideas and actions, the symbols are irrelevant, and we don't have to surrender them to the bad guys' claims.
Do you have a coherent principle that separates one from the other?
No, you are just giving them the power to actually do stuff without looking like they are doing stuff.
As long as they talk the talk, they don't have to walk the walk. You don''t have to actually care about issues, that's what the DEI department is for.
Tattoos can change, too. If I had a tattoo like that, but had come to see the error of my ways, I would have it removed, or if that was too costly and time-consuming (it takes a year or so of painful, expensive, periodic laser treatments to remove a tattoo), I'd have a tattoo artist cover it up with something else.
There's no excuse for keeping a neo-Nazi tattoo if you stop being a neo-Nazi and realize that neo-Nazis are disgusting people.