zlacker

[parent] [thread] 3 comments
1. jerf+(OP)[view] [source] 2024-09-16 15:17:26
I am at a complete loss as to how you get from "Of course Amazon's chip advantage is their own chip consumption and scale in AWS" to "being dismissive of Amazon's developer advocacy".

If they're doing better than the others, good for them. It still blindingly obvious that having the biggest cloud is a huge advantage for chip design and to successfully exploit savings because of that scale, not some sort of super amazing secret just now being revealed by IEEE.

replies(1): >>alephn+1o
2. alephn+1o[view] [source] 2024-09-16 17:31:15
>>jerf+(OP)
Both Google and MS had advantages that Amazon did not have in the mid-2010s in the ML space.

Google had the advantage of owning the entire ML and Infra stack (TensorFlow, K8s, BERT, CNCF) and Microsoft had an inbuilt advantage in research communities thanks to MS Research's outsized impact in fundamental ML research.

At the time, the Annapurna Labs acquisition was seen as a massive coin-toss because IBM went down a similar path a decade before and failed.

replies(1): >>dh2022+s31
◧◩
3. dh2022+s31[view] [source] [discussion] 2024-09-16 21:13:12
>>alephn+1o
"Microsoft had an inbuilt advantage in research communities thanks to MS Research's outsized impact in fundamental ML research"

I thought for a few minutes and I could not come up with an example of an ML technology that originated at MS Research and then spread outside MSFT. Care to give some examples? Thanks!

replies(1): >>alephn+ZK1
◧◩◪
4. alephn+ZK1[view] [source] [discussion] 2024-09-17 03:07:30
>>dh2022+s31
> Care to give some examples

In the 2010s they were the leader in NLP and the precursor of LLMs like GPT3/3.5/4/4o

Machine Translation with Human Parity (2018) - https://arxiv.org/abs/1803.05567

MT-DNN (2019) - https://arxiv.org/abs/1901.11504

MASS (2019) - https://arxiv.org/abs/1905.02450

VALL-E (2023) - https://arxiv.org/abs/2301.02111

VALL-E 2 (2024) - https://arxiv.org/abs/2406.05370

While OpenAI was the first to monetize an LLM at scale via ChatGPT, it's still the early stages of this field, and there is a lot of innovation that can still be leveraged, especially in non-English language modeling, machine translation, text-to-speech, etc.

It's in this segment that Microsoft Research shines moreso than even Google Research let alone other organizations because of their strong NLP background in Chinese (Microsoft Research Asia), South Asian languages (Microsoft Research India), Arabic (Microsoft Research's older work during the Iraq War), etc.

[go to top]