zlacker

[return to "Elon Musk sues Sam Altman, Greg Brockman, and OpenAI [pdf]"]
1. 1vuio0+vn2[view] [source] 2024-03-02 01:59:41
>>modele+(OP)
"In March 2023, OpenAI released its most powerful language model yet, GPT-4. GPT-4 is not just capable of reasoning. It is better at reasoning than average humans. It scored in the 90th percentile on the Uniform Bar Exam for lawyers. It scored in the 99th percentile on the GRE Verbal Assessment. It even scored a 77% on the Advanced Sommelier examination."

One could argue a common characteristic of the above exams is that they each test memory, and, as such, one could argue that GPT-4's above-average performance is not necessarily evidence of "reasoning". That is, GPT-4 has no "understanding" but it has formidable reading speed and retention (memory).

While preparation for the above exams depends heavily on memorisation, other exams may focus more on reasoning and understanding.

Surely GPT-4 would fail some exams. But when it comes to GPT-4's exam performance, only the positive results are reported.

https://freeman.vc/notes/reasoning-vs-memorization-in-llms

◧◩
2. romwel+Bo2[view] [source] 2024-03-02 02:11:22
>>1vuio0+vn2
>Surely GPT-4 would fail some exams

Some? It does hilariously badly on basic math.

With confidence, though.

◧◩◪
3. MacsHe+Nt2[view] [source] 2024-03-02 03:19:38
>>romwel+Bo2
GPT-4 with code interpreter is better at math than elite Math undergrads.
◧◩◪◨
4. b-side+tk3[view] [source] 2024-03-02 14:29:29
>>MacsHe+Nt2
You must be using a different GPT-4 than me. I recently tried to get it to reason about probability distributions arising from combining multiple probability distributions and it immediately started hallucinating.
◧◩◪◨⬒
5. DirkH+VR4[view] [source] 2024-03-03 05:32:29
>>b-side+tk3
Enable the code interpreter. It isn't enabled by default.
[go to top]