I know that but try and tell it to the average user. Even many of my techie colleagues aren't game to make changes to their phones for fear of losing some beloved feature. Frankly, I'm amazed at how tolerant people are to this level of surveillance.
That said, much can and does go wrong, resurrecting bricked phones seems to be a pastime of mine. As you know, whether one can decouple Google's spyware subsystem easily or not depends on the phone. If you can't gain access to the OS then it's not possible to roll one's own ASOP-based image or use some other one.
These days, many manufacturers are making it harder and harder to bypass security features, unlock the boot loader and install custom ROMs. Nevertheless I won't buy a phone without first checking whether I can install a custom ROM and it's definitely harder now than it was say five years ago.