zlacker

[parent] [thread] 2 comments
1. Within+(OP)[view] [source] 2024-01-07 09:43:47
The apples-to-apples comparison would be comparing an LLM with Leela with search turned off (only using a single board state)

According to figure 6b [0] removing MCTS reduces Elo by about 40%, scaling 1800 Elo by 5/3 gives us 3000 Elo which would be superhuman but not as good as e.g. LeelaZero.

[0]: https://gwern.net/doc/reinforcement-learning/model/alphago/2...

replies(2): >>edgyqu+TO >>sscg13+ui1
◧
2. edgyqu+TO[view] [source] 2024-01-07 17:18:16
>>Within+(OP)
I don’t think 3000 is superhuman though, it’s peak human as iirc magnus had an Elo of 3000 at one point
◧
3. sscg13+ui1[view] [source] 2024-01-07 20:32:00
>>Within+(OP)
Leela policy is around 2600 elo, or around the level of a strong grandmaster. Note that Go is different from chess since there are no draws, so skill difference is greatly magnified. Elo is always a relative scale (expected score is based on elo difference) so multiplication should not really make sense anyways.
[go to top]