zlacker

[parent] [thread] 43 comments
1. breadw+(OP)[view] [source] 2024-05-14 23:40:25
Ilya knows how ChatGPT works. Any company that hires him will be able to catch up with ChatGPT.
replies(8): >>threes+m >>mvkel+O5 >>xyzzy1+ec >>tootie+ge >>nothro+Ze >>gordon+9f >>khazho+6n >>menaci+ho
2. threes+m[view] [source] 2024-05-14 23:44:47
>>breadw+(OP)
You will still need the compute resources of Microsoft, Meta etc.

And they have their own people who equally know how LLMs work.

Even raising funds is not a certainty given that VCs are becoming more cautious with AI as they realise it's now a platform fight between the mega corporations.

replies(4): >>freshp+M2 >>vanjaj+K3 >>zeroCa+a4 >>ignora+S9
◧◩
3. freshp+M2[view] [source] [discussion] 2024-05-15 00:09:37
>>threes+m
> You will still need the compute resources of Microsoft, Meta etc.

2-3 years ago sure, but now? Maybe he knows how to reduce that need by order of magnitude.

◧◩
4. vanjaj+K3[view] [source] [discussion] 2024-05-15 00:17:49
>>threes+m
yep, and the puzzle isnt 'make chatgtp again' its 'catch up to chatgpt' which is a whole other puzzle
◧◩
5. zeroCa+a4[view] [source] [discussion] 2024-05-15 00:22:14
>>threes+m
Plenty of companies have compute, but everyone is barely catching up to GPT-4 over a year after release. I'm sure places like Meta would love to unlock the details of what makes it so good.
replies(1): >>Curiou+6c
6. mvkel+O5[view] [source] 2024-05-15 00:38:25
>>breadw+(OP)
Catching up right now is not a matter of tech innovation, but raw energy and compute.

Of course, the next -revolution- in AI could very well come from Ilya. But why would he bestow that honor to anyone? He can self fund it if he wants. It's an R&D project, not a scaling problem.

replies(6): >>int_19+na >>anvuon+Ja >>aether+Fb >>bmitc+0d >>whimsi+hd >>robbom+ti
◧◩
7. ignora+S9[view] [source] [discussion] 2024-05-15 01:14:02
>>threes+m
> Even raising funds is not a certainty given that VCs are becoming more cautious

You'd think any investor would be utterly stupid to not make an exception for Ilya, regardless.

replies(1): >>callal+Na
◧◩
8. int_19+na[view] [source] [discussion] 2024-05-15 01:17:42
>>mvkel+O5
If that is the case, why hasn't Google caught up yet?
replies(7): >>Barrin+rc >>gnarba+tc >>nunez+Jc >>whimsi+od >>paulpa+zf >>jacobs+9g >>mvkel+vj
◧◩
9. anvuon+Ja[view] [source] [discussion] 2024-05-15 01:20:51
>>mvkel+O5
I'd argue it's mainly a matter of people. Otherwise Meta, Amazon, Google, etc would've released their versions like yesterday.
◧◩◪
10. callal+Na[view] [source] [discussion] 2024-05-15 01:21:31
>>ignora+S9
That’s not such a given. There are a toooon of deep pockets in this space already and absolutely zero path to profitable products for any of them.
◧◩
11. aether+Fb[view] [source] [discussion] 2024-05-15 01:32:09
>>mvkel+O5
I mean, that's probably 90% or 95% true, but the remaining 5-10% is almost certainly worth a $100M offer to Sutskever from Google or Meta (or possibly Amazon or Apple).
◧◩◪
12. Curiou+6c[view] [source] [discussion] 2024-05-15 01:37:16
>>zeroCa+a4
There's no mystery, they just have a lot of actual user data, so they've been able to refine the question answering behavior. They've also baked some common problem solving strategies like chain of thought into the model via training.
13. xyzzy1+ec[view] [source] 2024-05-15 01:38:16
>>breadw+(OP)
If his concern was irresponsible AI proliferation, too much commercial focus and wanting to move more carefully, accelerating competition doesn't seem like it would align with his goals.
◧◩◪
14. Barrin+rc[view] [source] [discussion] 2024-05-15 01:40:07
>>int_19+na
probably because they have identified (correctly) that slowly integrating AI into their existing products that make them hundreds of billions is smarter than just burning that money to get upvoted on HN. Tall trees catch too much wind, ancient Chinese proverb. If you can afford to being second is usually less painful than being first.
replies(1): >>A4ET8a+Aj
◧◩◪
15. gnarba+tc[view] [source] [discussion] 2024-05-15 01:40:19
>>int_19+na
Google is ideologically compromised which ties them into knots and creates endless internal struggles.

this led to the release of Gemini which was absurdly biased when viewed by the typical American.

◧◩◪
16. nunez+Jc[view] [source] [discussion] 2024-05-15 01:43:43
>>int_19+na
Because Google is huge. Presumably the same reason why Microsoft finds it easier to shovel billions into OpenAI instead of doing it in-house.
◧◩
17. bmitc+0d[view] [source] [discussion] 2024-05-15 01:46:02
>>mvkel+O5
> Catching up right now is not a matter of tech innovation, but raw energy and compute.

I don't really understand that. ChatGPT is not all that impressive as an actual tool for many things. It doesn't really seem to matter to me what energy and compute is going into it. Will it make it actually work?

replies(1): >>mvkel+sk
◧◩
18. whimsi+hd[view] [source] [discussion] 2024-05-15 01:48:42
>>mvkel+O5
it’s a matter of data
◧◩◪
19. whimsi+od[view] [source] [discussion] 2024-05-15 01:49:19
>>int_19+na
my guess is that a significantly underrated reason is that Google holds itself to stricter data compliance standards than OAI
replies(1): >>elphin+me
20. tootie+ge[view] [source] 2024-05-15 01:57:34
>>breadw+(OP)
If he really is quitting over ethics he isn't going to Google or Meta. He'll probably go to Anthropic or academia. But who knows.
◧◩◪◨
21. elphin+me[view] [source] [discussion] 2024-05-15 01:59:01
>>whimsi+od
That is openly laughable. Incognito mode claims, just one of a long list of examples of this politicized monopoly engaging in criminal behavior.
replies(2): >>whimsi+ej >>astran+7N
22. nothro+Ze[view] [source] 2024-05-15 02:06:09
>>breadw+(OP)
Everyone knows how chatGPT works.
23. gordon+9f[view] [source] 2024-05-15 02:08:31
>>breadw+(OP)
first Andrej and now Ilya. Talent exodus at OpenAI?
◧◩◪
24. paulpa+zf[view] [source] [discussion] 2024-05-15 02:13:21
>>int_19+na
Wrong leadership at every level: Sundar as CEO, Prabhakar as Search SVP, Sissie as Assistant/Gemini VP. Maybe they should hire Ilya instead?
◧◩◪
25. jacobs+9g[view] [source] [discussion] 2024-05-15 02:18:58
>>int_19+na
Is Google significantly behind? I would say from a technological perspective, they're very closely trailing or even surpassing OpenAI in many ways; they've built a formidable ChatGPT competitor in Gemini—not to mention they have a huge home court advantage with billions of people on Search, Chrome, Android, and G Suite. If you zoom out a bit, it's not really a fair fight between the two. More likely, Apple and Microsoft win against Google because they use/buy OpenAI.
replies(2): >>duerra+4p >>int_19+JE
◧◩
26. robbom+ti[view] [source] [discussion] 2024-05-15 02:45:47
>>mvkel+O5
I feel like the lowest hanging fruit right now lies with the UI and already established techniques for reducing latency and making the experience smoother.

Just executing the client right would give someone a competitive advantage right now...

◧◩◪◨⬒
27. whimsi+ej[view] [source] [discussion] 2024-05-15 02:54:36
>>elphin+me
I don’t think that’s a fair take nor does it really disprove mine.

The compliance regime at big corps is definitely more sophisticated than at a player like OAI

◧◩◪
28. mvkel+vj[view] [source] [discussion] 2024-05-15 02:57:49
>>int_19+na
Because their weakness is in focus and execution, not resources.

If they focused all of their energy on, simply, a frontier AI model, and not trying to shoehorn a half-complete model into all of their products, there is no doubt they would be ahead.

But this is the innovator's dilemma, and why it is that startups are the disruptors.

Big companies move slow and lack focus. Small companies move fast and can only focus on one thing.

replies(1): >>miki12+ip
◧◩◪◨
29. A4ET8a+Aj[view] [source] [discussion] 2024-05-15 02:58:50
>>Barrin+rc
There are clear advantages and disadvantages to that approach, but assuming this is the correct choice ( seems plausible, but I am not automatically convinced ), is that the actual reason for being as behind as they are?

I would argue that is not the case. I won't re-list some of the reasons other posters mentioned, which, based on past year, appear more likely ( decisions hamstrung by corporate committees, data governance bureaucracy, and last, but not least, ideology focus ). Leadership that is actually focused on 'delivering value to the shareholder' or not being worried about first mover advantage seems only a part of it.

edit: added first mover wording

◧◩◪
30. mvkel+sk[view] [source] [discussion] 2024-05-15 03:08:50
>>bmitc+0d
If you consider the best frontier LLM in the world "not that impressive," I'm curious to know what you think is
replies(1): >>menaci+No
31. khazho+6n[view] [source] 2024-05-15 03:37:39
>>breadw+(OP)
I don't think it's good to lionize people like that, like some kind of tech Übermensch. He neither knows everything about ChatGPT, nor is he the only person there to know a whole lot about it.
32. menaci+ho[view] [source] 2024-05-15 03:52:57
>>breadw+(OP)
I think what is probably very stressful about this space is virtually everyone knows how ChatGPT works. It is not a theoretical leap. It's actually fairly predictable how this shakes out, and OpenAI is pretty vulnerable.

an LLM is a curiosity without user data, anyone with a big silo of data can put out something years behind frontier and still instantly see huge usage. No one wants to go to AI, they want AI to come to them, unless OpenAI can stake a claim in a super novel way they're the Dropbox.

It's not like someone is going to use insider OpenAI knowledge to build an LLM so advanced you switch email, phone, or ERP providers

replies(3): >>ramraj+jq >>visarg+pq >>astran+xN
◧◩◪◨
33. menaci+No[view] [source] [discussion] 2024-05-15 03:57:18
>>mvkel+sk
I think the point is not "chatGPT isn't good", it's that no one is beating physics, you can rough out the coefficients on a napkin, and they're playing chicken with who wants to set more piles of money on fire.

It _is_ the best frontier LLM in the world, and virtually the entire global population of people who care about that are in this thread

replies(1): >>ralfd+3K
◧◩◪◨
34. duerra+4p[view] [source] [discussion] 2024-05-15 04:01:13
>>jacobs+9g
Technical chops has never been Google's problem. Gemini for all the hype Google has thrown at it, has continued the recurring trend of G not knowing how to win with their product launches.
◧◩◪◨
35. miki12+ip[view] [source] [discussion] 2024-05-15 04:02:49
>>mvkel+vj
and politics, although they seem to have woken up (no pun intended) and started de-politicizing.
◧◩
36. ramraj+jq[view] [source] [discussion] 2024-05-15 04:16:21
>>menaci+ho
No they don’t; if they did Google and Meta would have put out offerings that objectively beat OpenAI. However (barring temporary lapses) they’ve stayed ahead of the curve. Someone who constantly thinks low of their competition or people they hate is bound to fail.
replies(1): >>menaci+ur
◧◩
37. visarg+pq[view] [source] [discussion] 2024-05-15 04:17:06
>>menaci+ho
Last time such a departure led to the creation of Anthropic (Dario Amodei), they even equalled GPT-4's performance
replies(1): >>menaci+Vr
◧◩◪
38. menaci+ur[view] [source] [discussion] 2024-05-15 04:31:13
>>ramraj+jq
First, "objectively beat" doesn't matter. Comparative LLM performance means more or less nothing without application.

Let's say Google's LLM has slightly poorer reasoning, and you have to be really clear when you tell it to delete your old emails. What are you going to do, go to ChatGPT and have it very eloquently walk you through how to manually delete your emails?

But the idea that Google, the terrifying nation-state, who covetously gobbled up most of the bright minds of a generation, just couldn't fathom RLHF, means you think there is some inherent magic at OpenAI.

I use OpenAI's models a lot, obviously they're great, but whole-thread-as-context as a product is not a product, and Google has people plural who could execute that from first principles.

◧◩◪
39. menaci+Vr[view] [source] [discussion] 2024-05-15 04:36:14
>>visarg+pq
I think it's less that only 4 wizards on earth can create it, and more that hardly anyone wants to.

If Apple, Google, or MS are integrating LLMs, they can't get away with "you can talk to this website". The intersection of "has the cash to train a LLM" and "LLM is itself the entire offering" is very small

◧◩◪◨
40. int_19+JE[view] [source] [discussion] 2024-05-15 07:06:51
>>jacobs+9g
When you push either model to its limits, yes, Google is pretty far behind GPT-4 with Gemini.
◧◩◪◨⬒
41. ralfd+3K[view] [source] [discussion] 2024-05-15 08:02:34
>>menaci+No
This will change when the rumors turn out to be true and Apples Siri is powered by OpenAI and a billion people have a working conversational AI in their pocket.
replies(1): >>menaci+7X1
◧◩◪◨⬒
42. astran+7N[view] [source] [discussion] 2024-05-15 08:29:16
>>elphin+me
Cynicism is just giving yourself an excuse to be wrong because it sounds cooler.
◧◩
43. astran+xN[view] [source] [discussion] 2024-05-15 08:34:29
>>menaci+ho
> It's not like someone is going to use insider OpenAI knowledge to build an LLM so advanced you switch email, phone, or ERP providers

If you mean the other kind of ERP they will definitely do that; it's basically all /r/localllama is about.

◧◩◪◨⬒⬓
44. menaci+7X1[view] [source] [discussion] 2024-05-15 16:15:36
>>ralfd+3K
I agree that this will happen (I said in another post), but I can't for the life of me attach it to one of the points I made
[go to top]