>> I think one missing step is to fuse training and inference regime into one, just as in animals.
This has always been an important missing piece. Without it ChatGPT is just a natural language interface to the information it was trained on. Still useful but unable to learn (aside from context).