There are some good stuff on the software side that people mention, but a big one is the driver support. We would need device makers to upstream support so there is less worrying about reverse engineering or needing to run modified ROMs based on old builds. Or just publish specs on the hardware that is enough for implementation. Sure, you can buy a specific phone and run a de-googled android or linux, but that only really works for the hobbyist who wants to spend time doing this. Which makes it difficult to create a market that encourages developers of software to port their software or write new software. With out being able to broadly support devices, most people are gonna be better off running Google's android.