zlacker

"On a very fundamental level the LLM is a function from context to the next token but when you generate text there is a state as the context gets updated with what has been generated so far."

Its output is predicated upon its training data, not user defined prompts.

>>shaneb+(OP)
If you have some data and continuously update it with a function, we usually call that data state. That's what happens when you keep adding tokens to the output. The "story so far" is the state of an LLM-based AI.

>>chpatr+k1
'If you have some data and continuously update it with a function, we usually call that data state. That's what happens when you keep adding tokens to the output. The "story so far" is the state of an LLM-based AI.'

You're conflating UX and LLM.

>>shaneb+(OP)
> Its output is predicated upon its training data, not user defined prompts.

Prompts very obviously have influence on the output.

>>shaneb+i3
I never said LLMs are stateful.

>>alpaca+66
"Prompts very obviously have influence on the output."

The LLM is also discrete.

>>shaneb+i3
You're being pedantic. While the core token generation function is stateless, that function is not, by a long shot, the only component of an LLM AI. Every LLM system being widely used today is stateful. And it's not only 'UX'. State is fundamental to how these models produce coherent output.

>>danena+bu
"State is fundamental to how these models produce coherent output."

Incorrect.