Attention and scale is all you need
Anything else you do will be overtaken by LLM when it builds its internal structures
Well, LLM and MCTS
The rest is old news. Like Cyc