Yikes.
This technology definitely needs to be open source, especially if we get to the point of AGI. Otherwise Microsoft and OpenAI are going to exploit it for as long as they can get away with it for profit, while open source lags behind.
Reminds me of the moral principles that guided Zimmermann when he made PGP free for everyone: A powerful technology is a danger to society if only a few people possess it. By giving it to everyone, you even the playing field.
That's why we all have personal nukes, of course. Very safe
Sam Altman:"Development of superhuman machine intelligence is probably the greatest threat to the continued existence of humanity."
In the essay "Why You Should Fear Machine Intelligence" https://blog.samaltman.com/machine-intelligence-part-1
So, more than nukes then...
Elon Musk: "There’s a strong probability that it [AGI] will make life much better and that we’ll have an age of abundance. And there’s some chance that it goes wrong and destroys humanity."