Seriously, try it. Why don't LLMs get frustrated with you if you ask them the same question repeatedly? A human would. Why are LLMs so happy to give contradictory answers, as long as you are very careful not to highlight the contradictory facts? Why do earlier models behave worse on reasoning tasks than later ones? These are features nobody, anywhere understands. So why make the (imo phenomenally large) leap to "well, it's clearly just a brain"?
It is like someone inventing the aeroplane and someone looks at it and says "oh, it's flying, I guess it's a bird". It's not a bird!
To be fair, I have had a strong sense of Gemini in particular becoming a lot more frustrated with me than GPT or Claude.
Yesterday I had it ensuring me that it was doing a great job, it was just me not understanding the challenge but it would break it down step by step just to make it obvious to me (only to repeat the same errors, but still)
I’ve just interpreted it as me reacting to the lower amount of sycophancy for now
Of course they will quickly revert to self-anthropomorphizing language, even after promising that they won't ... because they are just pattern matchers producing the sort of responses that conforms to the training data, not cognitive agents capable of making or keeping promises. It's an illusion.
You can tell it 'you are a machine, respond only with computerlike accuracy' and that is you gaslighting the cloud of probabilities and insisting it should act with a personality you elicit. It'll do what it can, in that you are directing it. You're prompting it. But there is neither a person there, nor a superintelligent machine that can draw on computerlike accuracy, because the DATA doesn't have any such thing. Just because it runs on lots of computers does not make it a computer, any more than it's a human.
We tried to mimic birds at first; it turns out birds were way too high-tech, and too optimized. We figured out how to fly when we ditched the biological distraction and focused on flight itself. But fast forward until today, we're reaching the level of technology that allows us to build machines that fly the same way birds do - and of such machines, it's fair to say, "it's a mechanical bird!".
Similarly, we cracked computing from grounds up. Babbage's difference engine was like da Vinci's drawings; ENIAC could be seen as Wright brothers' first flight.
With planes, we kept iterating - developing propellers, then jet engines, ramjets; we learned to move tons of cargo around the world, and travel at high multiples of the speed of sound. All that makes our flying machines way beyond anything nature ever produced, when compared along those narrow dimensions.
The same was true with computing: our machines and algorithms very quickly started to exceed what even smartest humans are capable of. Counting. Pathfinding. Remembering. Simulating and predicting. Reproducing data. And so on.
But much like birds were too high-tech for us to reproduce until now, so were general-purpose thinking machines. Now that we figured out a way to make a basic one, it's absolutely fair to say, "I guess it's like a digital mind".
Consider that we have recordings of Brent Spiner covered in white paint and wearing yellow contact lenses claiming to have no emotions, not because he didn't, but because he was playing a role, which is also something we know LLMs can do.
So we don't know for sure if LLMs do or don't have qualia, irregardless of what they say, and won't until we have a more concrete idea of what the mechanism is behind that sense of the phrase "mental state" so we can test for their presence or absence.
Um, that's what I said.
And of course we know that LLMs don't have qualia. Heck, even humans don't have qualia: https://web.ics.purdue.edu/~drkelly/DennettQuiningQualia1988...