zlacker

The guy with the "Bad universal priors and notions of optimality", which did to Hutter's MIRI program what Gödel did to Hilbert's program.

>>vinter+(OP)
Any chance you can eli5? I'm familiar with the Godel/Hilbert side but not the relationship to these developments.

>>debate+m1
Oops, I thought there was something odd, I got my rationality acronyms mixed up. Hutter's program was called AIXI (MIRI was the research lab).

Here is Leike's paper, coauthored with Hutter:

https://arxiv.org/abs/1510.04931

They can probably sum it up in their own paper better than I can, but AIXI was supposed to be a formalized, objective model of rationality. They knew from the start that it was uncomputable, but I think they hoped to use it as a sort of gold standard that you could approach.

But then it turned out that the choice of Turing machine, which can be (mostly) ignored for Kolmogorov complexity, can not be ignored in AIXI at all.