zlacker

>>homarp+(OP)
I mean, this seems obvious to me. How would the model predict the next move WITHOUT calculating the board state first? Yes, by memorization, but memorization hypothesis is easily rejected by comparison to training dataset in this case.

It is possible the model calculates an approximate board state, which is different from the board state but equivalent for most games, but not all games. It would be interesting to train adversarial policy to check this. From KataGo attack we know this does happen for Go AIs: Go rules have a concept of liberty, but so called pseudoliberty is easier to calculate and equivalent for most cases (but not all cases). In fact, human programmers also used pseudoliberty to optimize their engines. Adversarial attack found Go AIs also use pseudoliberty.

>>sanxiy+M61
It’s one thing to think it’s obvious, but quite another to prove it. I think this is the true value of this kind of work, is that it’s helping to decipher what these models are actually doing. Far too often we hear “NNs / LLMs are black boxes” as if that’s the end of the conversation.