Why AM and Eurisko Appear to Work (1983) [pdf]

>>zetaly+(OP)
Context: AM (Automated Mathematician) is a program that starts with a knowledge base of set theory and heuristics and builds its way up to natural numbers, Goldbach's conjecture (depending on how you interpret the print-outs), etc. Eurisko takes AM one step further by having heuristics to guide the development of new heuristics.

I read this because I'm working on a history/review of AM and Eurisko. It's one of the most fascinating episodes in the history of AI: a good old fashioned symbolist AI program that has all these impressive success stories, and defines the SOTA. Source code was never published and it was never reproduced. Eliezer Yudkowsky said[0] of Eurisko that it "may still be the most sophisticated self-improving AI ever built". Like something out of a Borges story!

Further reading for those interested:

  AM: An artificial intelligence approach to discovery in mathematics as heuristic search.
    Douglas B. Lenat. 1976.
    https://stacks.stanford.edu/file/druid:sb448rj9905/sb448rj9905.pdf

  The Nature of Heuristics I
    Douglas B. Lenat. 1982.
    http://blog.funcall.org/docs/am-eurisko/Heuristics_I.pdf

  The Nature of Heuristics II
    Douglas B. Lenat. 1983.
    http://blog.funcall.org/docs/am-eurisko/Heuristics_II.pdf

  The Nature of Heuristics III
    Douglas B. Lenat. 1983.
    http://blog.funcall.org/docs/am-eurisko/Eurisko_Heuristics_III.pdf

  On the thresholds of knowledge.
    Douglas B. Lenat, Edward A. Feigenbaum. 1991.
    http://blog.funcall.org/docs/am-eurisko/On_the_thresholds_of_knowledge.pdf

  AM: A Case Study in AI Methodology
    G.D. Ritchie, F.K. Hanna. 1984.
    https://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.89.4342&rep=rep1&type=pdf

AM has been reimplemented[1] in Prolog.

[0]: https://www.lesswrong.com/posts/rJLviHqJMTy8WQkow/recursion-...

[1]: https://github.com/akkartik/am-utexas/tree/master

>>zetaly+R6
Did Eurisko ever have any replicable success stories?

The only success story of any kind I've heard, repeatedly, were the two Traveller tournaments. But Lenat's telling of that story is disputed by the people who were involved in that community, and doesn't make much sense in some ways. And obviously it wasn't replicable either.

So honestly my assumption for the last couple of decades has been that Eurisko was all smoke and mirrors.

>>jsnell+89
I thought Lenat was pretty up front about what Eurisko did and why it was nearly a one-trick pony. What I gathered from his writings was that he essentially used a Genetic Algorithm (which was novel at the time as it would have been a very early, possibly the first, implementation) to modify the Lisp code. So it was essentially randomly swapping out tiny snippets (i.e. Lisp atoms) of the code and managed to achieve good results because of the impedance match between the Lisp code and the problem domain.

So while I wouldn't say it was smoke and mirrors, Eurisko was both novel (for the GA approach) and disappointing (it had nothing resembling understanding of the problem) at the same time.

zlacker