zlacker

>>vforgi+(OP)
Not the first time that OpenAI has claimed their technology is so good it's dangerous. (From early 2019: https://techcrunch.com/2019/02/17/openai-text-generator-dang...) This is the equivalent of martial artists saying that their hands have to be registered as deadly weapons.

>>xnx+102
50% of AI researchers think there’s a greater than 10% chance that AI causes human extinction. It’s not only openAI and sam who think this is dangerous.

>>HDThor+w62
They have no real models to base their predictions on. It's speculation at the level of "what happens if an immovable meets an unstoppable force". No rigor in it.

>>Satam+ok2
What qualifies in your view as a "real model"? Do we need to empirically observe the human race going extinct first?

>>mitthr+DN2
No but we should still be level headed. It's like arguing that because 1) fire kills and 2) spreads, that the whole earth will thus be engulfed in fire and everyone will die.

If there's no real model behind this, I can argue just as well that a sufficiently intelligent AGI will be able to protect me from any harm because it's so smart and powerful.

And newsflash, if nothing changes, we're all going to die anyway. As it is, our existence is quite limited and it is only through constant creation of unnatural contraptions that we have managed to improve our quality of life.

>>Satam+MM5
OK. Well, it turns out that the rationale behind AI risk is a bit more sophisticated than that.

>>mitthr+on8
Not ok. Show me evidence that the AI panic is driven by such accepted and sophisticated models.

>>Satam+2gb
Take your pick?

https://yoshuabengio.org/2023/05/22/how-rogue-ais-may-arise/

https://intelligence.org/files/AlignmentHardStart.pdf

https://www.youtube.com/watch?v=pYXy-A4siMw

https://europepmc.org/article/med/26185241

https://en.wikipedia.org/wiki/Superintelligence:_Paths,_Dang...

I could keep going but the reading on this spans for tens of thousands of pages of detailed reasoning and analysis, including mathematical proofs and lab-scale experiments.