zlacker

[return to "We gave 5 LLMs $100K to trade stocks for 8 months"]
1. bcrosb+l2[view] [source] 2025-12-04 23:20:57
>>cheese+(OP)
> Grok ended up performing the best while DeepSeek came close to second. Almost all the models had a tech-heavy portfolio which led them to do well. Gemini ended up in last place since it was the only one that had a large portfolio of non-tech stocks.

I'm not an investor or researcher, but this triggers my spidey sense... it seems to imply they aren't measuring what they think they are.

◧◩
2. IgorPa+Z3[view] [source] 2025-12-04 23:28:43
>>bcrosb+l2
Yeah I mean if you generally believe the tech sector is going to do well because it has been doing well you will beat the overall market. The problem is that you don’t know if and when there might be a correction. But since there is this one segment of the overall market that has this steady upwards trend and it hasn’t had a large crash, then yeah any pattern seeking system will identify “hey this line keeps going up!” Would it have the nuance to know when a crash is coming if none of the data you test it on has a crash?

It would almost be more interesting to specifically train the model on half the available market data, then test it on another half. But here it’s like they added a big free loot box to the game and then said “oh wow the player found really good gear that is better than the rest!”

Edit: from what I causally remember a hedge fund can beat the market for 2-4 years but at 10 years and up their chances of beating the market go to very close to zero. Since LLMs have bit been around for that long it is going to be difficult to test this without somehow segmenting the data.

◧◩◪
3. stonem+rt2[view] [source] 2025-12-05 17:02:14
>>IgorPa+Z3
Would that work for LLMs though? They hypothetically trained on news papers from the second half of the data so they have knowledge of "future" events.
[go to top]