zlacker

It can only be "super unlikely" for an AI to "run away and do it's own thing" when we actually know how to align it.

Which we don't.

So we're not aligning it with corporate boards yet, though not for lack of trying.

(While LLMs are not directly agents, they are easy enough to turn into agents, and there's plenty of people willing to do that and disregard any concerns about the wisdom of this).

So yes, the crapshoot is exactly what everyone in AI alignment is trying to prevent.

(There's also, confusingly, "AI safety", which includes alignment but also covers things like misuse, social responsibility, and so on)

>>ben_w+(OP)
"Run away" AI is total science fiction - i.e, not anything happening in the foreseeable future. That's simply not how these systems work. Any looming AI threat will be entirely the result of deliberate human actions.

>>root_a+m4
We've already had robots "run away" into a water feature in one case and a pedestrian pushing a bike in another, the phrase doesn't only mean getting paperclipped.

And for non-robotic AI, also flash-crashes on the stock market and that thing with Amazon book pricing bots caught up in a reactive cycle that drove up prices for a book they didn't have.

>>ben_w+xd
> the phrase doesn't only mean getting paperclipped.

This is what most people mean when they say "run away", i.e. the machine behaves in a surreptitious way to do things it was never designed to do, not a catastrophic failure that causes harm because the AI did not perform reliably.

>>root_a+7m
Every error is surreptitious to those who cannot predict the behaviour of a few billion matrix operations, which is most of us.

When people are not paying attention, they're just as dead if it's Therac-25 or Thule airforce base early warning radar or an actual paperclipper.

>>ben_w+Lv
No. Surreptitious means done with deliberate stealth to conceal your actions, not a miscalculation that results in a failure.

>>root_a+pw
To those who are dead, that's a distinction without a difference. So far as I'm aware, none of the "killer robots gone wrong" actual sci-fi starts with someone deliberately aiming to wipe themselves out, it's always a misspecification or an unintended consequence.

The fact that we don't know how to determine if there's a misspecification or an unintended consequence is the alignment problem.

>>ben_w+Ez
"Unintended consequences" has nothing to do with AI specifically, it's a problem endemic to every human system. The "alignment" problem has no meaning in today's AI landscape beyond whether or not your LLM will emit slurs.

>>ben_w+Ez
To those who are dead, it doesn't matter if there was a human behind the wheel, or a matrix