zlacker

Judging from this interview, I wouldn't hold my breath hoping for more openness. Ilya seems to be against open sourcing models on the grounds that they may be too powerful. Good thing no one asked him to invent a wheel, after all people could travel too fast for their own safety.

>>victor+(OP)
Maybe. We’re also not open sourcing DNA from viruses, how to build nuclear weapons or 3D printing weapons.

I think there is an argument to be made that not every powerful LLM should be open source. But yes- maybe we’re worried about nothing. On the other hand, these tools can easily spread misinformation, increase animosity, etc, Even in todays world.

I come from the medical field, and we make risk-analyses there to dictate how strict we need to tests things before we release it in the wild. None of this exists for AI (yet).

I do think that focus on alignment is many times more important than chatgpt stores for humanity though.

>>MVisse+h5
Huh? We absolutely have open source virus genome sequences and 3D printed gun plans.

>>MVisse+h5
Nuclear weapons are open sourced already. The trick was to acquire the means to make it without being sanctioned to hell.

>>MVisse+h5
Actually the genome for viruses, and bacteria, does seem to be open. Here is an FTP server where you can download a bunch of different diseases.

https://ftp.ncbi.nih.gov/genomes/genbank/

>>m-ee+zf
Fair point. I think the thrust of the argument still stands. Open source is generally a fantastic principle but it has its limits. I.e. we probably shouldn't open source bomb designs or superviruses.

>>ALittl+Il
That's true. There are many other viruses that we don't publish for good reasons though.

>>victor+(OP)
Yeah, these well meaning safety / human alignment ideas sound more like centrally planned communism. In theory, good for everyone, in practice bad for everyone.

Interestingly socialist Europe(45% of gdp) and even capitalistic usa (25%) collect and redistribute more in tax revenue than Russia (10%) and China (12%). Numbers from memory maybe slightly off.

The flaw in communism was the central planning. The flaw in ai safety / alignment is also the central planning. Capitalism redistributed more wealth to the poor. Decentralized ai will distribute more benefits to humans than a centralized ai, even if it’s openly planned.