zlacker

>>amrrs+(OP)
All the environments the test (Tower of Hanoi, Checkers Jumping, River Crossing, Block World) could easily be solved perfectly by any of the LLMs if the authors had allowed it to write code.

I don't really see how this is different from "LLMs can't multiply 20 digit numbers"--which btw, most humans can't either. I tried it once (using pen and paper) and consistently made errors somewhere.

>>thomas+kb1
> I don't really see how this is different from "LLMs can't multiply 20 digit numbers"--which btw, most humans can't either. I tried it once (using pen and paper) and consistently made errors somewhere.

People made missiles and precise engineering like jet aircraft before we had computers, humans can do all of those things reliably just by spending more time thinking about it, inventing better strategies and using more paper.

Our brains weren't made to do such computations, but a general intelligence can solve the problem anyway by using what it has in a smart way.

>>Jensso+Fe1
Some specialized people could probably do 20x20, but I'd still expect them to make a mistake at 100x100. The level we needed for space crafts was much less than that, and we had many levels of checks to help catch errors afterwards.

I'd wager that 95% of humans wouldn't be able to do 10x10 multiplication without errors, even if we paid them $100 to get it right. There's a reason we had to invent lots of machines to help us.

It would be an interesting social studies paper to try and recreate some "LLMs can't think" papers with humans.

>>thomas+jh1
I don't think you got @Jensson's point.

With enough effort and time we can arrive at a perfect solution to those problems without a computer.

This is not a hypothetical, it was like that for at least hundreds of years.

>>morale+Rz2
With enough time and effort you can build an entire science of how arbitrarily complex computations can be done with pen and paper without errors in an arbitrarily long amount of time.

But then you're not measuring the ability to perform the calculations, but the ability to invent the methods that make the calculation possible.