Maybe, but the GPL explicitly permits this by only requiring source to be distributed to those who receive the software.
If, instead, the GPL stated that source must be available to everybody when software is distributed to anyone, we maybe wouldn't have this RHEL situation? What would we lose if that were the case?
I think the main reason the requirement to publish the sources is limited to the person who receives the binary is simply practical. For one, there is basically no way to sue as a third party to a contract - even if the contract required you to publish all of your code openly, someone who didn't receive the binary can't really have standing to sue if you just don't publish it. Also, at the time the GPL was created, sending source code carried some measurable cost, so making it public would have been at least mildly expensive.