We currently make around 1 TW of photovoltaic cells per year, globally. The proposal here is to launch that much to space every 9 hours, complete with attached computers, continuously, from the moon.
edit: Also, this would capture a very trivial percentage of the Sun's power. A few trillionths per year.
Think about it. Elon conjures up a vision of the future where we've managed to increase our solar cell manufacturing capacity by two whole orders of magnitude and have the space launch capability for all of it along with tons and tons of other stuff and the best he comes up with is...GPUs in orbit?
This is essentially the superhero gadget technology problem, where comic books and movies gloss over the the civilization changing implications of some technology the hero invents to punch bad guys harder. Don't get me wrong, the idea of orbiting data centers is kind of cool if we can pull it off. But being able to pull if off implies an ability to do a lot more interesting things. The problem is that this is both wildly overambitious and somehow incredibly myopic at the same time.
One of the things space has going for it is abundant cheap energy in the form of solar power. What can you do with megawatts of power in space though? What would you do with it? People have thought about beaming it back to Earth, but you'd take a big efficiency hit.
AI training needs lots of power, and it's not latency sensitive. That makes it a good candidate for space-based compute.
I'm willing to believe it's the best low-hanging fruit at the moment. You don't need any major technological advances to build a proof-of-concept. Whether it's possible for this to work well enough that it's actually cheaper than an equivalent terrestrial datacenter now or in the near future is something I can't answer.
You do - cooling those datacenters in space is an unsolved problem.