The drivers; last I heard, literally every Android device on the market was using a forked kernel in order to support its hardware. And Google keeps trying things to improve that situation, but...
https://lwn.net/Articles/680109/ was ~9 years ago and since then not even Google themselves have managed to ship a device running a mainline kernel. Supposedly it should get better with their latest attempt to just put drivers and user space, but 1. I haven't heard of any devices actually shipping with an unmodified kernel, probably because 2. AIUI that doesn't cover all drivers anyways.