https://www.metaculus.com/questions/3479/date-weakly-general...
But also, some of the magic in having good enough pretrained representations is that you don’t need to train them further for downstream tasks, which means non-differentiable tasks like logic could soon become more tenable.