My guess is that modern hardware is too complicated for one hacker to write reliable drivers. That wasn't the case back in the 90-s, when Linux matured. So we are at mercy of hardware manufacturers and they happened to not be interested in open upstreamed drivers.
On any PC, you can still use BIOS/UEFI services to get a basic framebuffer and keyboard input. You cannot do that on embedded ARM devices - you need to get several layers into the graphics stack to have a framebuffer. I tried it on the PinePhone, using existing source code as a reference, and the furthest I got was sending commands from the video port to the LCD controller and then not having an oscilloscope to see if the LCD controller replied back.
Having basic framebuffer in BIOS/UEFI is neat for toy OSes, but not very relevant for something practical. You gotta need proper driver for GPU. And if you're just starting, UART console is actually more preferable way to interact with board, IMO.
Modern hardware has turned our operating systems into isolated "user OS" nodes in the schematics, completely sandboxed away from the real action. Our operating systems don't really operate systems anymore.