zlacker

>>i13e+(OP)
Google has figured out how to scale search to the current number of requests per second

Scaling a ChatGPT like product to that level would be challenging I assume - and extremely expensive. Is that correct?

>>farees+OT
Going rate is about 3 cents per LLM query, so at Google scale of 100k qps, it would be ~$300MM/day. However you probably need to send only about 1/10 of current queries to LLM (as most are still not suitrable for LLM to asnwer) so that would cost only $30MM/day or about $9B/year which is peanuts for Google.