zlacker

If you look at the paper, they only claim LLM can correct the errors if the mistake location is given. And the mistake finding part is not yet solved.

>>ming03+(OP)
They don't correct errors even then. They just generate something which sounds like what one might say in a conversation when constrained not to express the error. If there's essentially just one option for that, its the correct one - but then it's like telling someone that the answer to a yes/no question is not the one they generated. Not much "error correction" to do then.

>>einpok+di
Yep. Where you can see them really get tripped up is if there's multiple "levers" or points for potential contradiction. Lots of dependent clauses, chains of predicates that all have to line up for something to make sense. When they get one item wrong, they don't "see" the consequences for the others. And if you get them to correct one, they'll then often turn around and mess up the others.

Because at no point is the "mind" involved doing a step by step reduction of the problem. It doesn't do formal reasoning.

Humans usually don't either, but they can almost all do a form of it when required to. Either under the assistance of a teacher, or in extremis when they need to. We've all had the experience of being flustered, taking a deep breath, and then "working through" something. After spending time with GPT, etc it becomes clear they're not doing that.

It's not that reasoning comes intrinsic to all human thoughts -- we're far lazier than that -- but when we need to, we can usually do it.