I didn't construct some arbitrary personal bar. I simply acknowledged that edge cases exist that reasonable people might feel necessitate community action as a matter of self preservation. That doesn't undermine the general principle.
At the end of the day we're discussing social standards so there aren't going to be any airtight logical arguments and the edges will inevitably be blurry. If you adopt an extremist mindset you'll be able to rationalize just about anything. That doesn't mean you're actually in the right though.
Now, it's our job, when we see the arc not bending the right way (or fast enough) to do something, but I think to avoid allowing the arc to have levers is not doing ourselves any favors.
And with all the talk of cancel culture (not government action, but just all private citizen action), I've actually seen very few examples of it resulting in something that I consider unacceptable. Note, I'd consider physical threats outside the bounds of cancel culture -- those are just physical threats.