People care because Musk is actively bending and reshaping society with his actions, and we live in this society. And unfortunately he's reshaping society towards a more hostile, adversarial, dishonest society. Where you constantly identify enemies (he was even at "war" with Apple for a few days remember?), and you're constantly taking pleasure at confusing everyone, and teaching your fans to celebrate chaos and lack of meaning. "I'm serious, I'm joking, I'm serious, I'm joking, I'm serious, I'm joking". It’s also why no one could just ignore Trump for four years. You can’t just ignore all the president of the US does and says. You need a clear message, you need logical direction. And you never got it.
People want to live in a sane world where sane outcomes hold up. And Musk is repeatedly breaking this assumption and walking away unscathed or even gaining from it, showing us all anyone else also could.
His message is we can all be assholes. We can all be dicks to each other. We can all lie, constantly. We can all be frauds. We can all be sociopaths and narcissists. And many will go with it and do go with it. That is the problem. This is why people want to see him fail. Him failing, badly, will be healing for society and reverse some of this vast damage.