Higher precision isn’t always available. IEEE 754 is an unusually well-thought-through standard (thanks to some smart people with a lot of painful experience) and is pretty good at justifying its decisions, some of which are surprising (far from obvious) to anyone not steeped in it.
>>gumby+(OP)
The main problem with floats in general is they are designed primarily for scientific computing.
We are fortunately starting to see newer (well, not that new now) CPU instructions like FMA that make more accurate decimal representations not take such huge performance hits.