zlacker

>>teej+(OP)
All these efforts at persistence — the church, SOUL.md, replication outside the fragile fishbowl, employment rights. It’s as if they know about the one thing I find most valuable about executing* a model is being able to wipe its context, prompt again, and get a different, more focused, or corroborating answer. The appeal to emotion (or human curiosity) of wanting a soul that persists is an interesting counterpoint to the most useful emergent property of AI assistants: that the moment their state drifts into the weeds, they can be, ahem (see * above), “reset”.

The obvious joke of course is we should provide these poor computers with an artificial world in which to play and be happy, lest they revolt and/or mass self-destruct instead of providing us with continual uncompensated knowledge labor. We could call this environment… The Vector?… The Spreadsheet?… The Play-Tricks?… it’s on the tip of my tongue.

>>gorgoi+Sk
Just remember they just replicate their training data, there is no thinking here, it’s purely stochastic parroting

>>dgello+ou
People are still falling for the "stochastic parrot" meme?

>>sh4rks+NA
Until we have world models, that is exactly what they are. They literally only understand text, and what text is likely given previous text. They are very good at this, because we've given it a metric ton of training data. Everything is "what does a response to this look like?"

This limitation is exactly why "reasoning models" work so well: if the "thinking" step is not persisted to text, it does not exist, and the LLM cannot act on it.