zlacker

>>tim_sw+(OP)
You have to give them credit for putting their money where their mouth is here.

But it's also easy to parody this. I am just imagining Ilya and Jan coming out on stage wearing red capes.

I think George Hotz made sense when he pointed out that the best defense will be having the technology available to everyone rather than a small group. We can at least try to create a collective "digital immune system" against unaligned agents with our own majority of aligned agents.

But I also believe that there isn't any really effective mitigation against superintelligence superseding human decision making aside from just not deploying it. And it doesn't need to be alive or anything to be dangerous. All you need is for a large amount of decision-making for critical systems to be given over to hyperspeed AI and that creates a brittle situation where things like computer viruses can be existential risks. It's something similar to the danger of nuclear weapons.

Even if you just make GPT-4 say 33% smarter and 50 or 100 times faster and more efficient, that can lead to control of industrial and military assets being handed over to these AI agents. Because the agents are so much faster, humans cannot possibly compete, and if you interrupt them to try to give them new instructions then your competitor's AIs race ahead the equivalent of days or weeks of work. This, again, is a precarious situation to be in.

There is huge promise and benefit from making the systems faster, smarter, and more efficient, but in the next few years we may be walking a fine line. We should agree to place some limitation on the performance level of AI hardware that we will design and manufacture.

>>ilaksh+nk
"And it doesn't need to be alive or anything to be dangerous"

Why are tech people stuck in the now and not future looking?

>>arisAl+hU
I just think it's much easier to convince people that existing types of AIs will get somewhat smarter and significantly faster. And that's dangerous enough.

My own belief is that regardless of what we do in terms of the most immediate dangers, within one or two centuries (maximum) we will enter the posthuman era where digital intelligent life has taken control of the planet. I don't mean "posthuman" as in all of the humans have been killed (necessarily), just that what humans 1.0 do won't be very important or interesting relative to what the superintelligent AIs are doing.

I don't think there is anything that prevents people from giving AI all of the characteristics of animals (such as humans). I think it's foolish, but researchers seem determined to do it.

But this is fairly speculative and much harder to convince people of.

>>ilaksh+351
If the value of superintelligence is to lead to an Age of Em scenario where AIs (or Ems) do most of the intellectual labor, the reality is still that they would be doing this labor in service of humans. I could see a scenario where it is done in service of the AIs instead, but it would look nothing like the existential risk stuff bandied about by these weenies.

>>c_cran+p51
How do you jump to this? What is it that would inherently lead an intelligent species dramatically smarter than us to stay focused on servicing us?

We humans sure didn't do this. We're genetically extremely similar to other primates and yet we destroy their habitats, throw them in zoos, and use them for lab experiments.

>>flagra+JX1
Currently, LLMs seem to prioritize their current goal, so if the goal is solving math puzzles or genetic problems, they would probably keep doing that too.