One day, the rapid advancement of AI via LLMs will slow down and attention will again return to logical reasoning and knowledge representation as championed by the Cyc Project, Cycorp, its cyclists and Dr. Doug Lenat.
Why? If NN inference were so fast, we would compile C programs with it instead of using deductive logical inference that is executed efficiently by the compiler.
This is the definition of a strawman. Who is claiming that NN inference is always the fastest way to run computation?
Instead of trying to bring down another technology (neural networks), how about you focus on making symbolic methods usable to solve real-world problems; e.g. how can I build a robust email spam detection system with symbolic methods?
What's the point of all that data collecting dust and accomplishing not much of anything?
Perhaps LLMs can automate in large part the manual operations of building a future symbolic knowledge base organized by a universal upper ontology. Considering the amazing emergent features of sufficiently-large LLMs, what could emerge from a sufficiently large, reflective symbolic knowledge base?
I’m making a sloppy version that will contain all the symbols needed to run a multi-unit building.
Neural and symbolic AI will eventually merge. Symbolic models bring much needed efficiency and robustness via regularization.
If you read PAIP today, the most likely reason is that you want a master class in Lisp programming and/or want to learn a lot of tricks for getting good performance out of complex programs (which used to be part of AI and is in many ways being outsourced to hardware today).
None of this is to say you shouldn't read PAIP. You absolutely should. It's awesome. But its role is different now.
Other parts like coding an Eliza chatbot are indeed outdated. I have read AIMA and followed a long course that used it, but I didn't really like it. I found it too broad and shallow.
This is hugely problematic. If you get the premises wrong, many fallacies will follow.
LLMs can play many roles around this area, but their output cannot be trusted with significant verification and validation.
> This is the definition of a strawman.
(Actually, it is an example of a strawman.) Anyhow, rather than a strawman, I'd rather us get right into the fundamentals.
1. Feed-forward NN computation ('inference', which is an unfortunate word choice IMO) can provably provide universal function approximation under known conditions. And it can do so efficiently as well, with a lot of recent research getting into both the how and why. One "pays the cost" up-front with training in order to get fast prediction-time performance. The tradeoff is often worth it.
2. Function approximation is not as powerful as Turing completeness. FF NNs are not Turing complete.
3. Deductive chaining is a well-studied, well understood area of algorithms.
4. But... modeling of computational architectures (including processors, caches, busses, and RAM) with sufficient detail to optimize compilation is a hard problem. I wouldn't be surprised if this stretches these algorithms to the limit in terms of what developers will tolerate in terms of compile times. This is a strong incentive, so I'd expect there is at least some research that pushes outside the usual contours here.
I have two concerns. First, just after pointing out a logical fallacy from someone else, you added a fallacy: the either-or fallacy. (One can criticize a technology and do other things too.)
Second, you selected an example that illustrates a known and predictable weakness of symbolic systems. Still, there are plenty of real-world problems that symbolic systems address well. So your comment cherry-picks.
It appears as if you are trying to land a counter punch here. I'm weary of this kind of conversational pattern. Many of us know that tends to escalate. I don't want HN to go that direction. We all have varying experience and points of view to contribute. Let's try to be charitable, clear, and logical.
Well done, sir, well done.
My hunch is it emerges naturally out of the hierarchical generalization capabilities of multiple layer circuits. But then you need something to coordinate the acquired labels: a tweak on attention perhaps?
Another characteristic is probably some (limited) form of recursion, so the generalized labels emitted at the small end can be fed back in as tokens to be further processed at the big end.
This is unpersuasive without laying out your assumptions and reasoning.
Counter points:
(a) It would be unethical for such a knowledge base to be put out in the open without considerable guardrails and appropriate licensing. The details matter.
(b) Cycorp gets some funding from the U.S. Government; this changes both the set of options available and the calculus of weighing them.
(c) Not all nations have equivalent values. Unless one is a moral relativist, these differences should not be deemed equivalent nor irrelevant. As such, despite the flaws of U.S. values and some horrific decision-making throughout history, there are known worse actors and states. Such parties would make worse use of an extensive human-curated knowledge base.
That's one of the principal features of Cyc. It's carefully built by humans to be (essentially) logically sound. - so that inference can then be run through the fact base. Making that stuff logically sound made for a very detailed and fussy knowledge base. And that in turn made it difficult to expand or even understand for mere civilians. Cyc is NOT simple.
By contrast, LLMs for now are embarassing. With inconsistent nonsense provided within one answer or an answer not recognizing the context of the problem. Say, the work domain being a food label and the system not recognizing that or not staying within that.
I'd recommend that more people take a look and compare its approach against others. https://en.wikipedia.org/wiki/CycL is compact and worth a read, especially the concept of "microtheories".