zlacker

[parent] [thread] 1 comments
1. vineya+(OP)[view] [source] 2024-05-18 01:42:41
I understand this argument, but I can't help but feel we're all kidding ourselves assuming that their engineers are really living in the future.

The most obvious reason is costs - if it costs many millions to train foundation models, they don't have a ton of experiments sitting around on a shelf waiting to be used. They may only get 1 shot at the base-model training. Sure productization isn't instant, but no one is throwing out that investment or delaying it longer than necessary. I cannot fathom that you can train an LLM at like 1% size/tokens/parameters to experiment on hyper parameters, architecture, etc and have a strong idea on end-performance or marketability.

Additionally, I've been part of many product launches - both hyped up big-news-events and unheard of flops. Every time, I'd say that 25-50% of the product is built/polished in the mad rush between press event and launch day. For an ML Model, this might be different, but again see above point.

Sure products may be planned month/years out, but OpenAI didn't even know LLMs were going to be this big a deal in May 2022. They had GPT-2 and GPT-3 and thought they were fun toys at that time, and had an idea for a cool tech demo. I think that OpenAI (and Google, etc) are entirely living day-to-day with this tech like those of us on the outside.

replies(1): >>HarHar+KV
2. HarHar+KV[view] [source] 2024-05-18 14:43:02
>>vineya+(OP)
> I think that OpenAI (and Google, etc) are entirely living day-to-day with this tech like those of us on the outside.

I agree, and they are also living in a group-think bubble of AI/AGI hype. I don't think you'd be too welcome at OpenAI as a developer if you didn't believe they are on the path to AGI.

[go to top]