zlacker

A very interesting and worthwhile article (better than the comments here would suggest). However, I find it a bit of a pity that the author places so much emphasis on the assumption that the project has failed. The approach has not simply failed because the search for a solution has been going on for forty years. It took even more than forty years and costed at least as much before neural networks became really useful, and nobody would claim that the project failed because of that. And today's LLM are not really that intelligent yet. Maybe Cycorp's knowledge base will be made generally accessible at some point, so that it can be used to train LLMs. Perhaps then a greater benefit of this data will become apparent.

>>Rochus+(OP)
> on the assumption the project has failed.

My daughter's PhD thesis was largely negative results. Even if the project had failed, we could learn from it if it wasn't so secretive. It could be much more open without being OSS!

>>Rochus+(OP)
> Maybe Cycorp's knowledge base will be made generally accessible at some point, so that it can be used to train LLMs.

More likely, it will be made increasingly irrelevant as open alternatives to it are developed instead. The Wikipedia folks are working on some sort of openly developed interlingua that can be edited by humans, in order to populate Wikipedias in underrepresented languages with basic encyclopedic text. (Details very much TBD, but see https://en.wikipedia.org/wiki/Abstract_Wikipedia and https://meta.wikimedia.org/wiki/Abstract_Wikipedia ) This will probably be roughly as powerful as the system OP posits at some point in the article, that can generate text in both English and Japanese but only if fed with the right "common sense" to begin with. It's not clear exactly how useful logical inference on such statements might turn out to be, but the potential will definitely exist for something like that too, if it's found to be genuinely worthwhile in some way.

>>zozbot+Z
> made increasingly irrelevant as open alternatives to it are developed instead

Certainly interesting what these projects are going for, but it's unlikely an "open alternative", given that the degree of formalization and rigor achieved by Cyc's higher-order logic specification is likely not achievable by statistical learning, and a symbolic approach is barely achievable in a shorter time than Cyc.

>>smoyer+S
It is at least as important to know which approaches do not work, but this gets significantly less press, which is not that attractive for scientists in the age of "publish or perish".

>>zozbot+Z
https://www.wikidata.org/wiki/Wikidata:Main_Page, for those curious about the interlingua in question.

>>yellow+R7
Strictly speaking, Wikidata is an existing project which only provides a rather restrictive model for its assertions; they are not fully compositional, thus are quite far from being able to express general encyclopedic text, especially in a way that can be 'seamlessly' translated to natural language. It does provide a likely foundation for these further planned developments, though.

>>zozbot+Z
It would be very surprising if the results from this approach were superior to simply machine-translating the entries from another language—because e.g. English already has so much content and contributor activity, and LLMs are already very good at translating. I can’t imagine you’d get more than a fraction of people’s interest in authoring entries in this abstract language.

>>yowzad+ic
LLMs are good at translating between languages that have significant amounts of written content on the internet. There are few languages in this category that do not already have correspondingly large Wikipedias.

There are plenty of languages with millions of speakers that are only rarely used in writing, often because some other language is enforced in education. If you try to use an LLM to translate into such a language, you'll just get garbage.

It's very easy for a hand-crafted template to beat an LLM if the LLM can't do the job at all.

>>Rochus+(OP)
> Maybe Cycorp's knowledge base will be made generally accessible at some point

I would sooner hold my breath waiting for OpenAI to open up than Cycorp :)

> It took even more than forty years and costed at least as much before neural networks became really useful

The correct class of comparison to make with "neural networks" would be "symbolic AI" writ large. Symbolic AIs have been working quite well in some areas. Just not at all in terms of common sense reasoning, or anything approaching AGI.

If you want to keep "Cyc" in the comparison, then I would argue there is no comparison possible. Without exaggeration, there has never been a single project in AI as expensive as Cyc before 2020. Only with GPT-2 did the cost start to exceed the million USD mark. (Without exact figures, AlphaGo and Deep Blue probably also cost millions of dollars, but they unambiguously worked.)

It's also just not true that it took 40 years. Consider e.g. LeNet-5, which was up and running in 1998, and was used in ATMs to read real cheques. The main cost was 10 years of engineering stamina by LeCun's research group at Bell Labs. The finished version could be trained "for about 20 epoches over MNIST. It took 2 to 3 days of CPU time on a Silicon Graphics Origin 2000 server, using a single 200 MHz R10000 processor."

(1998 might technically be 40 years out from e.g. the inception of the perceptron in the 1950s, but if that is supposed to be our reference point for neural networks, then Cyc's reference point should be the inception of logical AIs in the same decade. And really, what use was Cyc in industry in 1998?)

>>smoyer+S
>> My daughter's PhD thesis was largely negative results.

Well, give us a link man! :)

>>YeGobl+hS
You're forcing the proud dad function: https://pubmed.ncbi.nlm.nih.gov/36995257/

>>smoyer+Pb3
Absolutely. The proud nerd dad function :D

Congratulations to your daughter for her PhD. I'm guessing she has got it by now.

Sonic hedgehog signalling pathway! And what a date to submit a thesis.

Why is that a negative result, btw?