A wealthy, powerful, influential celebrity figure saying something 'regrettable' on Twitter often has real-world consequences. If he'd posted a tweet that read something like "Upload a picture of you assassinating so-and-so and a Bitcoin address and I will send you $100k" and someone followed those instructions, that would be conspiracy to commit murder. If he'd posted "I feel that so-and-so's politics are misguided" that's totally reasonable free speech.
There's also a question of scale or exposure. We legally define rights by qualitative analyses. I feel strongly that as technology increases the power of an individual that quantitative analyses are relevant too. If someone's speech will be broadcasted to 400,000 or 40,000,000 followers, that's one thing, if it's said privately to 4 friends that's completely different.
Somewhere in the middle of these things there's a line between right and wrong. I'm quite confident that telling an audience of 400,000 that you want a few named people to "Die slow motherfuckers" is on the wrong side of that line.
Then why are we trusting the person to make a rational choice? By that logic, X should simply disallow using certain words when your audience is > 400,000. Because anyone can get incited for violence when they read certain words as you claim.