zlacker

[return to "Nvidia H100 GPUs: Supply and Demand"]
1. zoogen+Lo1[view] [source] 2023-08-01 15:27:47
>>tin7in+(OP)
The real gut-punch for this is a reminder how far behind most engineers are in this race. With web 1.0 and web 2.0 at least you could rent a cheap VPS for $10/month and try out some stuff. There is almost no universe where a couple of guys in their garage are getting access to 1000+ H100s with a capital cost in the multiple millions. Even renting at that scale is $4k/hour. That is going to add up quickly.

I hope we find a path to at least fine-tuning medium sized models for prices that aren't outrageous. Even the tiny corp's tinybox [1] is $15k and I don't know how much actual work one could get done on it.

If the majority of startups are just "wrappers around OpenAI (et al.)" the reason is pretty obvious.

1. https://tinygrad.org/

◧◩
2. tedivm+ur1[view] [source] 2023-08-01 15:37:13
>>zoogen+Lo1
I'd argue that you really don't need 1000+ H100s to test things out and make a viable product.

When I was at Rad AI we managed just fine. We took a big chunk of our seed round and used it to purchase our own cluster, which we setup at Colovore in Santa Clara. We had dozens, not hundreds, of GPUs and it set us back about half a million.

The one thing I can't stress enough- do not rent these machines. For the cost of renting a machine from AWS for 8 months you can own one of these machines and cover all of the datacenter costs- this basically makes it "free" from the eight month to three year mark. Once we decoupled our training from cloud prices we were able to do a lot more training and research. Maintenance of the machines is surprisingly easy, and they keep their value too since there's such a high demand for them.

I'd also argue that you don't need the H100s to get started. Most of our initial work was on much cheaper GPUs, with the A100s we purchased being reserved for training production models rapidly. What you need, and is far harder to get, is researchers who actually understand the models so they can improve the models themselves (rather than just compensating with more data and training). That was what really made the difference for Rad AI.

◧◩◪
3. somsak+vw1[view] [source] 2023-08-01 15:56:41
>>tedivm+ur1
keep in mind that if you're a bigger customer, aws discounts are huge (often >50% off of sticker). if the payback were 16 months instead of 8, it becomes a much tougher sell (esp with GPUs improving rapidly over time)
◧◩◪◨
4. tedivm+9z1[view] [source] 2023-08-01 16:05:47
>>somsak+vw1
AWS does not offer great discounts on the GPUs at this point, as they don't have nearly enough of them to meet demand. I'm no longer at a startup, and have worked at a couple of larger companies.

That said I'm mostly responding to the "two guys in a garage" comment with this. Larger companies are going to have different needs altogether.

[go to top]