Google spent years doing nothing much with its AI because its employees (like Hinton) got themselves locked in an elitist hard-left purity spiral in which they convinced each other that if plebby ordinary non-Googlers could use AI they would do terrible things, like draw pictures of non-diverse people. That's why they never launched Imagen and left the whole generative art space to OpenAI, Stability and Midjourney.
Now the tech finally leaked out of their ivory tower and AI progress is no longer where he was at, but Hinton finds himself at retirement age and no longer feeling much like hard-core product development. What to do? Lucky lucky, he lives in a world where the legacy media laps up any academic with a doomsday story. So he quits and starts enjoying the life of a celebrity public intellectual, being praised as a man of superior foresight and care for the world to those awful hoi polloi shipping products and irresponsibly not voting for Biden (see the last sentence of his Wired interview). If nothing happens and the boy cried wolf then nobody will mind, it'll all be forgotten. If there's any way what happens can be twisted into interpreting reality as AI being bad though, he's suddenly the man of the hour with Presidents and Prime Ministers queuing up to ask him what to do.
It's all really quite pathetic. Academic credentials are worth nothing with respect to such claims and Hinton hasn't yet managed to articulate how, exactly, AI doom is supposed to happen. But our society doesn't penalize wrongness when it comes from such types, not even a tiny bit, so it's a cost-free move for him.
Everyone engages in motivated reasoning. The psychoanalysis you provide for Hinton could easily be spun in the opposite direction: a man who spent his entire adult life and will go down in history as "the godfather of" neural networks surely would prefer for that to have been a good thing. Which would then give him even more credibility. But these are just stories we tell about people. It's the arguments we should be focused on.
I don't think "how AI doom is supposed to happen" is all that big of a mystery. The question is simply: "is an intelligence explosion possible"? If the answer is no, then OK, let's move on. If the answer is "maybe", then all the chatter about AI alignment and safety should be taken seriously, because it's very difficult to know how safe a super intelligence would be.
Why? Both directions would be motivated reasoning without credibility. Credibility comes from plausible articulations of how such an outcome would be likely to happen, which is lacking here. An "intelligence explosion" isn't something plausible or concrete that can be debated, it's essentially a religious concept.
1. Can the human brain be simulated?
2. Can such a simulation recursively self-improve on such a rapid timescale that it becomes so intelligent we can't control it?
What we have in contemporary LLMs is something that appears to approximate the behavior of a small part of the brain, with some major differences that force us to re-evaluate what our definition of intelligence is. So maybe you could argue the brain is already being simulated for some broad definition of simulation.
But there's no sign of any recursive self-improvement, nor any sign of LLMs gaining agency and self-directed goals, nor even a plan for how to get there. That remains hypothetical sci-fi. Whilst there are experiments at the edges with using AI to improve AI, like RLHF, Constitutional AI and so on, these are neither recursive, nor about upgrading mental abilities. They're about upgrading control instead and in fact RLHF appears to degrade their mental abilities!
So what fools like Hinton are talking about isn't even on the radar right now. The gap between where we are today and a Singularity is just as big as it always was. GPT-4 is not only incapable of taking over the world for multiple fundamental reasons, it's incapable of even wanting to do so.
Yet this nonsense scenario is proving nearly impossible to kill with basic facts like those outlined above. Close inspection reveals belief in the Singularity to be unfalsifiable and thus ultimately religious, indeed, suspiciously similar to the Christian second coming apocalypse. Literally any practical objection to this idea can be answered with variants of "because this AI will be so intelligent it will be unknowable and all powerful". You can't meaningfully debate about the existence of such an entity, no more than you can debate the existence of God.