zlacker

>>mef+(OP)
All this interview proves is that Chomsky has fallen far, far behind how AI systems work today and is retreating to scoff at all the progress machine learning has achieved. Machine learning has given rise to AI now. It can't explain itself from principles or its architecture. But you couldn't explain your brain from principles or its architecture, you'd need all of neuroscience to do it. Because the brain is digital and (probably) does not reason like our brains do, it somehow falls short?

While there's some things in this I find myself nodding along to in this, I can't help but feel it's an a really old take that is super vague and hand-wavy. The truth is that all of the progress on machine learning is absolutely science. We understand extremely well how to make neural networks learn efficiently; it's why the data leads anywhere at all. Backpropagation and gradient descent are extraordinarily powerful. Not to mention all the "just engineering" of making chips crunch incredible amounts of numbers.

Chomsky is extremely ungenerous to the progress and also pretty flippant about what this stuff can do.

I think we should probably stop listening to Chomsky; he hasn't said anything here that he hasn't already say a thousand times for decades.

>>titzer+S5
> The truth is that all of the progress on machine learning is absolutely science

It is not science, which is the study of the natural world. You are using the word "science" as an honorific, meaning something like "useful technical work that I think is impressive".

The reason you are so confused is that you can't distinguish studying the natural world from engineering.

>>foobar+kc
LLMs certainly aren't science. But there is a "science of LLMs" going on in, e.g., the interpretability work by Anthropic.