zlacker

[parent] [thread] 1 comments
1. boredh+(OP)[view] [source] 2023-03-01 19:38:08
> Why would we think they would go rogue in different ways

Their prompts would differ, depending on their use case. For ChatGPT, even a few words can effect a huge change in the personality it shows.

> Are there any others?

Both scenarios are vague enough for lots of uncertainty. If many AIs are around, perhaps they would see each other as bigger threats and ignore mankind. And different optimizing tasks might conflict with each other. There could be a paperclip recycler for every paperclip maker.

replies(1): >>adamsm+Eg
2. adamsm+Eg[view] [source] 2023-03-01 20:56:43
>>boredh+(OP)
Look into something called Instrumental Convergence. The TLDR is that basically any advanced AI system with some set of high level goals is going to converge on a set of sub goals (self preservation, adding more compute, improving it's own design, etc.) that all lead to bad things for humanity. I.e paperclip maximizers might realize that Humans getting in the way of it's paperclip maximizing is a problem so it decides to neutralize them. In order to do so it needs to improve it's capabilities so works towards gathering more compute and improving it's own design. A Financial Trading AI realizes that it can generate more profit if it can gather more compute and improve it's design. An Asteroid Mining AI realizes it can build more probes if it had more compute to control more factories so it sets about gathering more compute and improving it's own design. Eliminating humans who may shut the AI off is often such a sub goal.
[go to top]