However, personal views aside, looking at it purely technically, it’s just a mindless token soup, that’s why I find it weird that even deeply technical people like Andrej Karpathy (there was a post made by him somewhere today) find it fascinating.
That’s why AI models, as they currently are, won’t ever be able to come up with anything even remotely novel.
I see no evidence for you magical ability to behave outside of being a function of context and memory.
You don’t think diffusion models are capable of novelty?
Visual cortex and computer vision show striking similarities, as do language processing.