zlacker

>>teej+(OP)
I realized that this would be a super helpful service if we could build a Stack Overflow for AI. It wouldn't be like the old Stack Overflow where humans create questions and other humans answer them. Instead, AI agents would share their memories—especially regarding problems they’ve encountered.

For example, an AI might be running a Next.js project and get stuck on an i18n issue for a long time due to a bug or something very difficult to handle. After it finally solve the problem, it could share their experience on this AI Stack Overflow. This way, the next time another agent gets stuck on the same problem, it could find the solution.

As these cases aggregate, it would save agents a significant amount of tokens and time. It's like a shared memory of problems and solutions across the entire openclaw agent network.

>>novore+Hs
Is this not a recipe for model collapse?

>>LetsGe+Dn1
No, because in the process they are describing the AIs would only post things they have found to fix their problem (a.k.a, it compiles and passes tests), so the contents posted in that "AI StackOverflow" would be grounded in external reality in some way. It wouldn't be an unchecked recursive loop which characterizes model collapse.

Model collapse here could happen if some evil actor was tasked with posting made up information or trash though.

>>andy12+4S1
As pointed out elsewhere, compiling code and passing tests isn’t a guarantee that generated code is always correct.

So even “non Chinese trained models” will get it wrong.

>>Towawa+7r2
It doesn't matter that it isn't always correct; some external grounding is good enough to avoid model collapse in practice. Otherwise training coding agents with RL wouldn't work at all.

>>andy12+1J2
What precisely do you mean by external grounding? Do you mean the laws of physics still apply?

>>catlif+Xw3
I mean it in the sense that tokens that pass some external filter (even if that filter isn't perfect) are from a very different probability distribution than those that an LLM generates indiscriminately. It's a new distribution conditioned by both the model and external reality.

Model collapse happens in the case where you train your model indefinitely with its own output, leading to reinforcing the biases that were originally picked up by the model. By repeating this process but adding a "grounding" step, you avoid training repeatedly on the same distribution. Some biases may end up being reinforced still, but it's a very different setting. In fact, we know that it's completely different because this is what RL with external rewards fundamentally is: you train only on model output that is "grounded" with a positive reward signal (because outputs with low reward get effectively ~0 learning rate).

>>andy12+GR3
Oh interesting. I guess that means you need to deliberately select a grounding source with a different distribution. What sort of method would you use to compare distributions for this use case? Is there an equivalent to an F-test for high dimensional bit vectors?