The problem is that people like Musk have spent ages arguing that banning fascists is bad because free speech absolutism is an important value. It turns out that free speech absolutism was never actually a value they cared about - the only thing that matters is that their guy is the one choosing the bans. If people like Musk had instead argued that platforming fascists is actually good this whole time then the discussion today would be different, but because they didn't want to publicly support fascists they had to fall back on the free speech absolutism argument, which has shattered into a million pieces.
Combine this with a general support of conservative fiscal policy as a wealthy business-owner and the libertarian ideals of a gen-X nerd who came of age during "information wants to be free" and obviously suffering from a compulsive social media addiction (pot calling kettle black here), and it's no surprise he's completely bought into "free speech conservatism", where slander and hatred are placed on even footing with legitimate political argument.