zlacker

>>tedsan+(OP)
This is a Military project. Have no doubts about it.

>>belter+X71
This is a money making scheme.

>>Gud+ie1
This has cosmological significance if it leads to superintelligence

>>arisAl+we1
It won't unless there's another (r)evolution in the underlying technology / science / algorithms, at this point scaling up just means they use bigger datasets or more iterations, but it's more finetuning and improving the existing output then coming up with a next generation / superintelligence.

>>Cthulh+al1
Okay, but let’s be pessimistic for a moment. What can we do if that revolution does happen, and they’re close to AGI?

I don’t believe the control problem is solved, but I’m not sure it would matter if it is.

>>Fillig+3o1
Being pessimistic, how come no human supergeniuses ever took over the world? Why didn't Leibniz make everyone else into his slaves?

I don't even understand what the proposed mechanism for "rouge AI enslaves humanity" is. It's scifi (and not hard scifi) as far as I can see.

>>ForHac+zo1
Once you have one AGI, you can scale it to many AGI as long as you have the necessary compute. An AGI never needs to take breaks, can work non-stop on a problem, has access to all of the world's information simultaneously, and can interact with any system it's connected to.

To put it simply, it could outcompete humanity on every metric that matters, especially given recent advancements in robotics.

>>Philpa+Zp1
...so it can think really hard all the time and come up with lots of great, devious evil ideas?

Again, I wonder why no group of smart people with brilliant ideas has unilaterally imposed those ideas on the rest of humanity through sheer force of genius.

>>ForHac+Us1
An equivalent advance in autonomous robotics would solve the force projection issue, if that's what you're getting at.

I don't know if this will happen with any certainty, but the general idea of commoditising intelligence very much has the ability to tip the world order: every problem that can be tackled by throwing brainpower at it will be, and those advances will compound.

Also, the question you're posing did happen: it was called the Manhattan Project.

>>Philpa+9u1
And if this whole exercise turns out to be a flop and gets us absolutely nowhere closer to AGI?

“AGI” has proven to be today’s hot marketing stunt for when you need to raise another round of cash and your only viable product is optimism.

Flying cars were just around the corner in the 60s, too.

>>redser+UJ1
This thread started from a deliberately pessimistic hypothetical of what happens if AGI actually manifests, so your comment is misplaced.