zlacker

You are arguing that a piece of software misses a metaphorical soul (something that cannot be measured but that humans uniquely have and nothing else does). That's an incredibly poor argument to make in a context where folks want interesting conversation. Religion (or religion-adjacent concepts such as this one) is a conversational nuke: It signals to anyone else that the conversation is over, as a discussion on religion cannot take forms that are fundamentally interesting. It's all opinion, shouted back and forth.

Edit: Because it is a prominent feature in the responses until now, I will clarify that there is an emphasis on "all" in "all opinion". As in, it is nothing but whatever someone believes with no foundation in anything measurable or observable.

>>krageo+(OP)
I didn’t read it as being a religious take. They appear to be referring more to embodiment (edit: alternatively, online/continual learning) which these models do not posses. When we start persisting recurrent states beyond the current session we might be able to consider that limited embodiment. Even still the models will have no direct experience interacting with the subjects of their conservations. Its all second hand from the training data.

>>krageo+(OP)
I find it ironic that you are expressing a strong opinion that opinions do not make good conversation. Philosophy is the highest form of interesting conversation, and it's right there with religion (possibly politics, too).

>>krageo+(OP)
Perhaps you have people around you who are not well suited to political, religious philosophical discussions or perhaps you don’t enjoy them / can’t entertain them.

Personally, I find the only interesting conversations technical or philosophical in nature. Just the other day, I was discussing with friends how ethics used to be a regular debated topic in society. Literally, every Sunday people would gather and discuss what it is to be a good human.

Today, we demonize one another, in large part because no one shares an ethical principal. No one can even discuss it and if they try, many people shut down the conversation (as you mentioned). In reality, it’s probably the only conversation worth having.

>>veheme+M2
Philosophy can be interesting if it is not baseless navel gazing (i.e. it is founded on observation and fact, and derives from there). The fact that I find that interesting is subjective, but that's not the meat of the post.

Religion is fundamentally folks saying "No, I'm right!" and nothing else. Sometimes it's dressed up a little. What could be interesting about that? You can hear such arguments in any primary school playground during recess.

>>letter+y4
A discussion about ethics must involve a discussion about the effects of a system of ethics on a group of people: This is a real-world effect that must have it's issues and priors spoken about, or you risk creating an ethical system for a group of people that will inevitably destroy them (which I would argue is bad, but I guess that is also debatable).

Such a discussion is about something tangible, and not purely about held opinion (i.e. you can go out and test it). I can see how someone might find that engaging. You are right that I usually do not (unless my conversational buddies have something novel to say about the subject, I find it extremely tedious). It is a good point, thank you.

>>soraki+72
Your own experience is also second hand, so what is left is the temporal factor (you experience and learn continuously and with a small feedback loop). I do not see how it can be the case that there is some sort of cutoff where the feedback loop is fast enough that something is "truly" there. This is a nebulous argument that I do not see ending when we actually get to human-equivalent learning response times, because the box is not bounded and is fundamentally based on human exceptionalism. I will admit I may be biased because of the conversations I've had on the subject in the past.

>>krageo+x7
Second hand may not have been the best phrasing on my part, I admit. What I mean is that the model only has textual knowledge in its dataset to infer what “basketball” means. It’s never seen/heard a game, even if through someone else’s eyes/ears. It has never held and felt a basketball. Even visual language models today only get a single photo right now. It's an open question how much that matters and if the model can convey that experience entirely through language.

There are entire bodies of literature addressing things the current generation of available LLMs are missing: online and continual learning, retrieval from short-term memory, the experience from watching all YouTube videos, etc.

I agree that human exceptionalism and vitalism are common in these discussions but we can still discuss model deficiencies from a research and application point of view without assuming a religious argument.

>>krageo+(OP)
It doesn't have to have a metaphorical (or metaphysical or w/e) soul, but at this point it does not have it's own 'opinion'. It will happily argue either way with only a light push, it talks because it is ordered to, not because it is trying to communicate information. This severely limits the kind of things it can do.

>>krageo+(OP)
I would argue (of course not seriously) about the opposite: ChatGPT has a metaphorical soul. What it learned very well is how to structure the responses so that they sound convincing - no matter how right or wrong they are. And that's dangerous.