zlacker

>>mef+(OP)
The fact that we have figured out how to translate language into something a computer can "understand" should thrill linguists. Taking a word (token) and abstracting it's "meaning" as a 1,000-dimension vector seems like something that should revolutionize the field of linguistics. A whole new tool for analyzing and understanding the underlying patterns of all language!

And there's a fact here that's very hard to dispute, this method works. I can give a computer instructions and it "understands" them in a way that wasn't possible before LLMs. The main debate now is over the semantics of words like "understanding" and whether or not an LLM is conscious in the same way as a human being (it isn't).

>>caliba+cd
Restricted to linguistics, LLM's supposed lack of understanding should be a non-sequitur. If the question is whether LLMs have formed a coherent ability to parse human languages, the answer is obviously yes. In fact not just human languages, as seen with multimodality the same transformer architecture seems to work well to model and generate anything with inherent structure.

I'm surprised that he doesn't mention "universal grammar" once in that essay. Maybe it so happens that humans do have some innate "universal grammar" wired in by instinct but it's clearly not _necessary_ to be able to parse things. You don't need to set up some explicit language rules or generative structure, enough data and the model learns to produce it. I wonder if anyone has gone back and tried to see if you can extract out some explicit generative rules from the learned representation though.

Since the "universal grammar" hypothesis isn't really falsifiable, at best you can hope for some generalized equivalent that's isomorphic to the platonic representation hypothesis and claim that all human language is aligned in some given latent representation, and that our brains have been optimized to be able to work in this subspace. That's at least a testable assumption, by trying to reverse engineer the geometry of the space LLMs have learned.

>>kracke+AG
Can LLMs actually parse human languages? Or can they react to stimuli with a trained behavioral response? Dogs can learn to sit when you say "sit", and learn to roll over when you say "roll over". But the dog doesn't parse human language; it reacts to stimuli with a trained behavioral response.

(I'm not that familiar with LLM/ML, but it seems like trained behavioral response rather than intelligent parsing. I believe this is part of why it hallucinates? It doesn't understand concepts, it just spits out words - perhaps a parrot is a better metaphor?)

>>0xbadc+GS
Dogs out of all creatures probably actually do have some parsing going on for human language. They learn it like we do, picking up context from the environment, actions, tone, facial expressions, body language, etc.

You can say 'what's that' in many different ways and a clever dog will react differently for each, even if it's the first time it's heard you say 'what's that?' In a scared tone it'll still react differently while knowing what you're asking.

They even do the cute head tilt when they're struggling to understand something.

I think people vastly underestimate the power of wetware and think animals and us are separated by a chasm, but I think it's a relatively small leap.

We base so much of our understanding of other creatures intelligence on their ability to communicate with us or express things in the ways we do. If elephants judged humans on their ability to communicate in infrasound to speak their names (yes they have names for each other) they'd wouldn't think too highly of us.

Sidenote but the latest I've heard is that elephants like us because they think we are cute.

>>Xss3+PW
Parsing? I don't think dogs "parse" language in a decompositional matter.