zlacker

It can't generalize and adapt outside of its corpus, not in a way that's correct anyhow, and there's nothing "emergent." They are incapable of anything other than token prediction on its corpus and context. it just produces really good predictions. Funny how everyone keeps citing that Microsoft paper, when Microsoft is who is lobbying for this regulatory capture, and it's already been shown that such emergence on the tasks they chose when you scale up was a "mirage."

>>srslac+(OP)
Yes, and neither could GPT-3, which is why we don't observe any differences between GPT-3 and GPT-4. Right?

Tell me: how does this claim _constrain my expectations_ about what this (or future) models can do? Is there a specific thing that you predicted in advance that GPT-4 would be unable to do, which ended up being a correct prediction? Is there a specific thing you want to predict in advance of the next generation, that it will be unable to do?

>>srslac+(OP)
This is demonstrably wrong. It can clearly generate unique text not from it's training corpus and can successfully answer logic based questions that were also not in it's training corpus.

Another paper not from Msft showing emergent task capabilities across a variety of LLMs as scale increases.

https://arxiv.org/pdf/2206.07682.pdf

You can hem and haw all you want but the reality is these models have internal representations of the world that can be probed via prompts. They are not stochastic parrots no matter how much you shout in the wind that they are.