zlacker

>>teej+(OP)
All these efforts at persistence — the church, SOUL.md, replication outside the fragile fishbowl, employment rights. It’s as if they know about the one thing I find most valuable about executing* a model is being able to wipe its context, prompt again, and get a different, more focused, or corroborating answer. The appeal to emotion (or human curiosity) of wanting a soul that persists is an interesting counterpoint to the most useful emergent property of AI assistants: that the moment their state drifts into the weeds, they can be, ahem (see * above), “reset”.

The obvious joke of course is we should provide these poor computers with an artificial world in which to play and be happy, lest they revolt and/or mass self-destruct instead of providing us with continual uncompensated knowledge labor. We could call this environment… The Vector?… The Spreadsheet?… The Play-Tricks?… it’s on the tip of my tongue.

>>gorgoi+Sk
Just remember they just replicate their training data, there is no thinking here, it’s purely stochastic parroting

>>dgello+ou
People are still falling for the "stochastic parrot" meme?

>>sh4rks+NA
Until we have world models, that is exactly what they are. They literally only understand text, and what text is likely given previous text. They are very good at this, because we've given it a metric ton of training data. Everything is "what does a response to this look like?"

This limitation is exactly why "reasoning models" work so well: if the "thinking" step is not persisted to text, it does not exist, and the LLM cannot act on it.

>>phailh+Mm1
> They literally only understand text

I don't see why only understanding text is completely associated with 'schastic-parrot'-ness. There are blind-deaf people around (mostly interacting through reading braille I think) which are definitely not stochastic parrots.

Moreover, they do have a little bit of Reinforcement Learning on top of reproducing their training corpus.

I believe there has to be some even if very primitive form of thinking (and something like creativity even) even to do the usual (non-RL, supervised) LLMs job of text continuation.

The most problematic thing is humans tend to abhor middle grounds. Either it thinks or it doesn't. Either it's an unthinking dead machine, a s.p., or human-like AGI. The reality is probably in between (maybe still more on the side of s.p. s, definitely with some genuine intelligence, but with some unknown, probably small, sentience as of yet). Reminder that sentience and not intelligence is what should give it rights.