While I find AI impressive, I think the demonstration proof that it isn't yet at that level is that ChaosGPT etc. have not already destroyed civilisation.
(OTOH, that someone made ChaosGPT and set it running, is reason to try to stop anyone publishing any better models until they can be proven safe: we don't want to find out something has passed this threshold, whatever that means, via it ending civilisation).