zlacker

AWS is so anti-customer with respect to GPUs right now.

They have the highest prices of any cloud. What happened to “your margin is my opportunity”?

And, as far as I know, customers are unable to allocate a VM with fewer than eight A100, H100, or H200 GPUs. (Please tell me how if I’m wrong.)

So, customers are incentivized to use other cloud products for GPUs in the short term.

They seem to be heavily invested in their own chips in the medium term.

>>rytill+(OP)
They're trying to sell you their managed product, i.e. bedrock.

>>rytill+(OP)
Meanwhile, I had a hard time last week getting a machine with 8 gpus from azure.

>>lpasse+621
Every customer of a 3rd party cloud provider is going to have a hard time getting Nvidia GPU compute from 3rd party cloud providers - not only is the amount of GPUs available limited, but Nvidia themselves are trying to spin up their own cloud provider offering, and unsurprisingly don't want to help a competitor.

>>rytill+(OP)
They typically come physically in 8s as an SXM5 tray.

It takes more effort to virtualize SXM5 than individual PCIe cards, so you find availability in 8s in VMs too regardless of provider

>>rytill+(OP)
They seem to be sold out of an extremely popular product and seem to be making good money doing so (per AWS’s results). How’s that “anti-customer?”

Also, in case you didn’t get the memo GPUs are hard to come by most places these days.

>>alephn+C81
> but Nvidia themselves are trying to spin up their own cloud provider offering, and unsurprisingly don't want to help a competitor

I thought Jensen recently said he does not want to offer their own cloud offering. He instead wants to focus on creating ready made solutions for cloud vendors to purchase & re-sell services with.

>>mattfe+id1
Fair point! I think that's a recentish pivot though (past 2-3 years). I vaguely remember that in the late 2010s they were building and testing DGX Cloud as it's own standalone offering, but I might be wrong and confusing some other offerings they were working on.