It's tempting to look at Moore's law and use say the development of the 8080, z-80 and 6502 in 1975 as an epoch. But it's hard to use that to get a visceral sense of how much things changed. I think RAM - in other words, available memory - may be more helpful, and it does relate in a distant way with model size and available GPU memory.
So the question is, if we surveyed a group of devs, engineers and computer scientists in 1975 and asked them to extrapolate and predict available RAM a few decades out, how well would their predictions map to reality?
In 1975 the Altair 8800 microcomputer with the 8080 processor had 8K of memory for the high end kit (4096 words).
8 years later, in 1983 the Apple IIe (which I learned to program on) had 64K RAM as standard, or 8 times the RAM.
13 years later in 1996, 16 to 32 MB was fairly commonplace in desktop PCs. That's 32,768K which is 4096 times the 8K available 21 years earlier.
30 years later in 2005, it wasn't unusual to find 1GB of RAM or 1,048,576K or 131,072 times 8K from 30 years earlier.
Is it realistic to expect a 1975 programmer, hardware engineer or computer scientist to predict that available memory in a desktop machine will be over 100,000 times greater 30 years in the future? We're not even taking into account moving from byte oriented CPUs to 32bit CPUs, or memory bandwidth.
2054 is 30 years in the future. It's going to fly by. I think given the unbelievable rate of change we've seen in the past, and how it accelerates, any prediction today from the smartest and most forward thinking people in AI will vastly underestimate what 2054 will look like.
Edit: 32bit CPU's not 64. Typo.