zlacker

It's more like you started a company to research new fuel sources and hired climate scientists to evaluate the environmental impact of those new fuel sources, but during the course of the research you invented a new internal combustion engine that operates 1000% more efficiently so you pivoted your entire business toward that, removing the need for climate scientists.

This is a tortured analogy, but what I'm getting at is, if OpenAI is no longer pursuing AGI/superintelligence, it doesn't need an expensive superintelligence alignment team.

>>mjr00+(OP)
You're much more confident than I am that the researchers at OpenAI (or anyone else currently alive) are masters of their craft to such an extent that they would even be able to predict whether the next big training run they do will result in a superintelligence or not. Another way of saying the same thing is to say that the only way anyone knows that GPT-4 is not dangerously capable is that it has been deployed extensively enough by now that if it was going to harm us, it would've done so by now: not even the researchers that designed and coded-up GPT-4 or watched it during training could predict with any confidence how capable it would be. For example, everyone was quite surprised by its being able to score in the 90th decile on a bar exam.

Also, even if they never produce a superintelligence, they are likely to produce insights that would make it easier for other teams to produce a superintelligence. (Since employees are free to leave OpenAI and join some other team, there is no practical way to prevent the flow of insights out of OpenAI.)

>>mjr00+(OP)
> if OpenAI is no longer pursuing AGI/superintelligence

What leads you to believe that's true?

>>marius+Q4
Microsoft dumping $10 billion into them to commercialize LLM tech, primarily.

>>marius+Q4
You can't build a business on AGI, which is an unbounded research project without any foreseeable end or path to revenue. However, LLMs definitely have some commercial value and OpenAI has a first-mover market advantage that they'd be insane to squander from a business perspective. I'm sure they will continue to do research in advancements of AI, but AGI is still science fiction.

>>mjr00+o7
If anything, in my opinion, the more runway they have, the better chances to actually hit an inflexion point in AGI development. But maybe you're right.

>>holler+K4
Why do they need to be 'masters of their craft' to place directional bets?

>>holler+K4
Call me uninformed, but I do not see a way forward where a statistical model trained to recognise relationships between words or groups of words, and has a front end coded to query that model could suddenly develop its own independence. That's a whole other thing, where the code to interact with it must allow for constant feedback loops of self.improvement and the vast amount of evolutionary activity that entails.

An interactive mathematical model is not going to run away on its own without some very deliberate steps to take it in that direction.

>>anakai+6g2
We have no idea how consciousness works. Just because you dont see a way forward doesnt mean its not there

>>holler+K4
As someone who has worked on LLMs somewhat extensively, the idea that we are going to accidentally make a superintelligence by that path is literally laughable.

>>Michae+Ca2
Hmm. It's hard for me to see why you think 'diectional bet' helps us understand the situation.

Certainly, the researchers want the model to be as useful as possible, so there we have what I would call a 'directional bet', but since usefulness is correlated with capability to potentially do harm (i.e., dangerousness) that bet is probably not what you are referring to.

>>anakai+6g2
You’re right. But are you saying LLMs couldn’t be a part of a more complex system similar to how our brain appears to be several integrated systems with special purpose and interdependence? I assume you’re not assuming everything is static and open ai is incapable of doing anything other offering incremental refinements in chatgpt? Just because they released X doesn’t mean Y+X isn’t coming. And we are talking about a longer game than “right this very second” - where do things go over 10 years? It’s not like open ai is going anywhere.

Maybe the guys who point out tar in tobacco is dangerous and nicotine is addictive maybe we shouldn’t add more for profit and such things would be useful just in case we get there.

But even if we don’t - an increasingly capable multimodal AI has a lot of utility for good and bad. Are we creating power tools with no safety? Or safety written by a bunch of engineers whose life experience extends to their PhD program at an exclusive school studying advanced mathematics? When their limited world collides with complex moral and ethical domains they don’t always have enough context to know why things are the way they are and our forefathers aren’t idiots. They often blunder into a mistake out of hubris.

Put it another way the chance they succeed is non zero. The possibility they succeed and they create a powerful tool that’s incredibly dangerous is non zero too. Maybe we should try to hedge that risk ?

>>HDThor+Bn2
I think the point was that on a purely technical level, the LLMs as currently used can’t do anything on their own. They only continue a prompt when given. It’s not like a LLM could “decide” to hack the NSA and publish the data tomorrow, because it determined that this would help humanity. The only thing it can do is try to make people do something when they read the responses.

>>fnordp+jY2
I was not saying that LLMs could not be part of a more complex system. What I was saying is that the more complex system is what likely needs to be the focus of discussion rather than the LLM itself.

Basically- the LLM won't run away on its own.

I do agree with a safety focus and guardrails. I dont agree with chicken little sky is falling claims.

>>echoan+RJ4
This is a good interpretation of the point I was getting at, yes.