zlacker

You are chaining some big assumptions together to make this conclusion. We can suppose AGI is around the corner, but these other assumptions are massive leaps that would need strong arguments to back them: - AGI thinks similar to humans - AGI knowing we are afraid of it will make it consider humans a threat

Unpacking that second point are the implications that: - AGI considering humans a threat is conditional on our fearing it - AGI seeing humans as a threat is the only reason it would harm humans

I feel like I can rule out these last 3 points just by pointing out that there are humans that see other humans as a threat even though there is not a display of fear. Someone could be threatening because of greed, envy, ignorance, carelessness, drugs, etc.

Also humans harm other humans all this time in situations where there was not a perceived threat. How many people have been killed by cigarettes? Car accidents? Malpractice?

And this is going off the assumption that AGI thinks like a human, which I'm incredibly skeptical of.

>>yeck+(OP)
I am fully aware that there are big assumptions.

But our most effective experiment so far is based on creating LLMs that try to act like humans. Specifically try to predict the next token that human speech would create. When AI is developed off of large scale models that attempt to imitate humans, shouldn't we expect that in some ways it will also imitate human emotional behavior?

What is "really" going on is another question. But any mass of human experience that you train a model on really does include our forms of irrationality in addition to our language and logic. With little concrete details for our speculation, this possibility at least deserves consideration.