Where will that glass ceiling be? Well of course it'll be just behind where OpenAI and other market leaders are at any given time. The argument will be made that because we know current offerings are safe, the ceiling should be just below current offerings.
So it becomes basically illegal to compete with market leaders, namely OpenAI/Microsoft and Google. Other huge corporations will be able to buy their way into the club but upstarts will find it nearly impossible.
Regulation will also require that everyone doing anything non-trivial report it to regulatory bodies so existing market incumbents can become aware of it. They can then prepare a strategy to respond either by lobbying to block it with regulation or preparing a market response ahead of time using massively greater resources. No stealth mode AI startups or other private efforts that OpenAI et. al. don't know about would be allowed.
The end result of this will be to push innovation in this area off shore and open source onto the dark web.