As humanity has struggled to understand the world, it has frequently given names to concepts that seem to matter, well before it is capable of explaining with any sort of precision what these things are, and what makes them matter - take the word 'energy', for example.
It seems clear to me that one must have these vague concepts before one can begin to to understand them, and also that it would be bizarre not to give them a name at that point - and so, at that point, we have a word without a locked-down definition. To insist that we should have the definition locked down before we begin to investigate the phenomenon or concept is precisely the wrong way to go about understanding it: we refine and rewrite the definitions as a consequence of what our investigations have discovered. Again, 'energy' provides a useful case study for how this happens.
A third point about the word 'energy' is that it has become well-defined within physics, and yet retains much of its original vagueness in everyday usage, where, in addition, it is often used metaphorically. This is not a problem, except when someone makes the lexicographical fallacy of thinking that one can freely substitute the physics definition into everyday speech (or vice-versa) without changing the meaning.
With many concepts about the mental, including 'reasoning', we are still in the learning-and-writing-the-definition stage. For example, let's take the definition you bring up: reasoning as good cognition. This just moves us on to the questions of what 'cognition' means, and what distinguishes good cognition from bad cognition (for example, is a valid logical argument predicated on what turns out to be a false assumption an example of reasoning-as-good-cognition?) We are not going to settle the matter by leafing through a dictionary, any more than Pedro Carolino could write a phrase book just from a Portugese-English dictionary (and you are probably aware that looking up definitions-of-definitions recursively in a dictionary often ends up in a loop.)
A lot of people want to jump the gun on this, and say definitively either that LLMs have achieved reasoning (or general intelligence or a theory of mind or even consciousness, for that matter) or that they have not (or cannot.) What we should be doing, IMHO, is to put aside these questions until we have learned enough to say more precisely what these terms denote, by studying humans, other animals, and what I consider to be the surprising effectiveness of LLMs - and that is what the interviewee in the article we are nominally discussing here is doing.
You entered this thread by saying (about the paper underlying an article in Ars Tech [1]) I’ll pop in with a friendly “that research is definitely wrong”. If they want to prove that LLMs can’t reason..., but I do not think there is anything like that claim in the paper itself (one should not simply trust what some person on HN says about a paper. That, of course, goes as much for what I say about it as what the original poster said.) To me, this looks like the sort of careful, specific and objective work that will lead to us a better understanding of our concepts of the mental.
The first three paragraphs you wrote very succinctly and obviously summarize the fundamental flaw of our modern science - that it can't make leaps, at all.
There is no leap of faith in science but there is science that requires such leaps.
We are stuck bc those most capable of comprehending concepts they don't understand and are unexplainable - they won't allow themselves to even develop a vague understanding of such concepts. The scientific method is their trusty hammer and their faith in it renders all that isn't a nail unscientific.
Admitting that they don't kno enough would be akin to societal suicide of their current position - the deciders of what is or isn't true, so I don't expect them to withhold their conclusions til they are more able to.
They are the "priest class" now ;)
I agree with your humble opinion - there is much more we could learn if that was our intent and considering the potential of this, I think we absolutely ought to make certain that we do everything in our power to attain the best possible outcomes of these current and future developments.
Transparent and honest collaboration for the betterment of humanity is the only right path to an AGI god - to oversimplify a lil bit.
Very astute, well formulated position, presented in accessible language and with humility even!
Well done.