zlacker

>>mef+(OP)
[flagged]

>>newAcc+56
From some Googling and use of Claude (and from summaries of the suggestively titled "Impossible Languages" by Moro linked from https://en.wikipedia.org/wiki/Universal_grammar ), it looks like he's referring to languages which violate the laws which constrain the languages humans are innately capable of learning. But it's very unclear why "machine M is capable of learning more complex languages than humans" implies anything about the linguistic competence or the intelligence of machine M.

>>Smaug1+b8
It doesn't, it just says that LLMs are not useful models of the human language faculty.

>>foobar+Fa
This is where I'm stuck.

For other commentators, as I understand it, Chomsky's talking about well-defined grammar and language and production systems. Think Hofstadter's Godel Escher Bach. Not "folk" understanding of language.

I have no understanding or intuition, or even a finger nail grasp, for how an LLM generates, seemingly emulating, "sentences", as though created with a generative grammar.

Is any one comparing and contrasting these two different techniques? Being noob, I wouldn't even know where to start looking.

I've gleaned that someone(s) are using LLM/GPT to emit abstract syntax trees (vs a mere stream of tokens), to serve as input for formal grammars (eg programming source code). That sounds awesome. And something I might some day sorta understand.

I've also gleaned that, given sufficient computing power, training data for future LLMs will have tokenized words (vs just character sequences). Which would bring the two strategies closer...? I have no idea.

(Am noob, so forgive my poor use of terminology. And poor understanding of the tech, too.)

>>specia+4h
I don't really understand your question but if a deep neural network predicts the weather we don't have any problem accepting that the deep neural network is not an explanatory model of the weather (the weather is not a neural net). The same is true of predicting language tokens.

>>foobar+Cj
Apologies, I don't know enough to articulate my question, which is probably nonsensical any way.

LLMs (like GPT) and grammars (like Backus–Naur Form) are two different kinds of generative (production) systems, right?

You've been (heroically) explaining Chomsky's criticism of LLMs to other noobs: grammars (theoretically) explain how humans do language, which is very different from how ChatGPT (stochastic parrots) do language. Right?

Since GPT mimics human language so convincingly, I've been wondering if there's any overlap of these two generative systems.

Especially once the (tokenized) training data for GPTs is word based instead of just snippets of characters.

Because I notice grammars everywhere and GPT is still magic to me. Maybe I'd benefit if I could understand GPTs in terms of grammars.