zlacker

Superintelligence (along with some definitions): https://en.wikipedia.org/wiki/Superintelligence

Also, "Dario Amodei says what he has seen inside Anthropic in the past few months leads him to believe that in the next 2 or 3 years we will see AI systems that are better than almost all humans at almost all tasks"

https://x.com/tsarnick/status/1881794265648615886

>>nopins+(OP)
Not saying you're necessarily wrong, but "Anthropic CEO says that the work going on in Anthropic is super good and will produce fantastic results in 2 or 3 years" it not necessarily telling of anything.

>>hatefu+M2
Dario said in mid-2023 that his timeline for achieving "generally well-educated humans" was 2-3 years. o1 and Sonnet 3.5 (new) have already fulfilled that requirement in terms of Q&A, ahead of his earlier timeline.

>>nopins+A3
I'm curious about that. Those models are definitely more knowledgeable than a well educated human, but so is Google search, and has been for a long time. But are they as intelligent as a well educated human? I feel like there's a huge qualitative difference. I trust the intelligence of those models much less than an educated human.

>>hatefu+S7
If we talk about a median well-educated human, o1 likely passes the bar. Quite a few tests of reasoning suggests that’s the case. An example:

“Preprint out today that tests o1-preview's medical reasoning experiments against a baseline of 100s of clinicians.

In this case the title says it all:

Superhuman performance of a large language model on the reasoning tasks of a physician

Link: https://arxiv.org/abs/2412.10849”. — Adam Rodman, a co-author of the paper https://x.com/AdamRodmanMD/status/186902305691786464

—-

Have you tried using o1 with a variety of problems?

>>nopins+S8
The paper you linked claims on page 10 that machines have been performing comparably on the task since 2012, so I'm not sure exactly what the paper is supposed to show in this context.

Am I to conclude that we've had a comparably intelligent machine since 2012?

Given the similar performance between GPT4 and O1 on this task, I wonder if GPT3.5 is significantly better than a human, too.

Sorry if my thoughts are a bit scattered, but it feels like that benchmark shows how good statistical methods are in general, not that LLMs are better reasoners.

You've probably read and understood more than me, so I'm happy for you to clarify.

>>hatefu+da
Figure 1 shows a significant improvement of o1-preview over earlier models.

Perhaps it’s better that you ask a statistician you trust.

>>nopins+Sa
The figure also shows that the non LLM algorithm from 2012 was as capable or more capable than a human: was it as intelligent as a well educated human?

If not, why is the study sufficient evidence for the LLM, but not sufficient evidence for the previous system?

Again, it feels like statistical methods are winning out in general.

> Perhaps it’s better that you ask a statistician you trust

Maybe we can shortcut this conversation by each of us simply consulting O1 :^)

>>nopins+A3
Can they do rule 110? If not, I don't think they're 'generally intelligent'.

>>nopins+A3
But there's 0 guarantee they are even capable of solving the rather large amount that covers the rest of a well-educated human.

>>hatefu+Xb
1) It’s an example of a domain an LLM can do better than humans. A 2012 system was not able to do myriad other things LLMs can do and thus not qualified as general intelligence.

2) As mentioned in the chart label, earlier systems require manual symptom extraction.

3) An important point well articulated by a cancer genomics faculty member at Harvard:

“….Now, back to today: The newest generation of generative deep learning models (genAI) is different.

For cancer data, the reason these models hold so much potential is exactly the reason why they were not preferred in the first place: they make almost no explicit data assumptions.

These models are excellent at learning whatever implicit distribution from the data they are trained on

Such distributions don’t need to be explainable. Nor do they even need to be specified

When presented with tons of data, these models can just learn, internalize & understand…..”

More here: https://x.com/simocristea/status/1881927022852870372?s=61&t=...

>>nopins+(OP)
Anthropic has to say this or Anthropic does not see their next funding round.