Definitely an out of this world idea. I wonder if their micro datacenter is going to be self-sufficient power wise using only solar energy? And how would they address the hardware failures that are likely when you train large language models at scale?
>>abetah+(OP)
Given the purported cost benefits in their whitepaper [1], hardware failures might be an irrelevant rounding error. They suggest something to the tune of 100x cheaper.
>>abetah+(OP)
The white paper back of the envelope calculations show a 4km x 4km solar panels and radiators are required for a 5 GW datacenter. I am not sure how the authors were not cracking up while writing that white paper.