zlacker

>>isaacf+(OP)
What I take away from listening to Sam talk is that in the beginning of Open AI, they didn’t think big compute would be as important as it has become.

It becomes very hard to not have profit incentives when you need to run gigantic supercomputers to push the technology forward. This explains the MS partnership and need to generate profit to fund the training and running of future models.

This doesn’t explain everything, but makes sense to this layman

>>stephe+8s1
I'm flabbergasted that someone would think that AI research wouldn't cost a lot in computer resources.

Next Sam will tell that is that farmers need a lot of land to grow crops.

I'm calling BS on this. It's an excuse not an explanation.

>>Teever+2t1
I think it's the difference between spending 10's of millions and a billion on computing resources.

When he started OpenAI, no one was spending anywhere near that much on compute. He should have listened to gwern.

>>JamesB+kv1
Exactly. The idea that massive compute would lead to massive improvement was non-obvious.

It was quite reasonable to think that there would be rapidly diminishing returns in model size.

Wrong, in hindsight, but that's how hindsight is.

>>crazyg+PK1
Google and all the big players in AI have known they need tons of data and hence compute power for processing it, for a very long time, way before OpenAI even existed. Anyone getting involved in that game would have definitely known.