zlacker

Terrifying that people are creating financial models with AI when they don’t have the skills to verify the model does what they expect

>>simmer+(OP)
They have an excel sheet next to it - they can test it against that. Plus they can ask questions if something seems off and have it explain the code.

>>martin+i
I'm not sure being able to verify that it's vaguely correct really solves the issue. Consider how many edge cases inhabit a "30 sheet, mind-numbingly complicated" Excel document. Verifying equivalence sounds nontrivial, to put it mildly.

>>simmer+(OP)
All we need is one major crash caused by AI to scare the capital owners. Then maybe us white collar workers can breath a bit for at least another few more years(maybe a decade+).

>>martin+i
> They have an excel sheet next to it - they can test it against that.

It used to be that we'd fix the copy-paste bugs in the excel sheet when we converted it to a proper model, good to know that we'll now preserve them forever.

>>simmer+(OP)
It's not terrifying at all, some shops will fail and some will succeed and in the aggregate it'll be no different for the rest of us

>>simmer+(OP)
I’m trying to learn rust coming from python (for fun). I use various LLM for python and see it stumble.

It is a beautiful experience to realize wtf you don’t know and how far over their skis so many will get trusting AI. The idea of deploying a rust project at my level of ability with an AI at the helm is is terrifying.

>>simmer+(OP)
Business as usual.

>>simmer+(OP)
If they have the skills to verify the Excel model then they can apply the same approach to the numbers produced by the AI-generated model, even if they can’t inspect it directly.

In my experience a lot of Excel models aren’t really tested, just checked a bit and them deemed correct.

>>nebula+03
A decade+ is wishful copium.

>>nebula+03
All we need is one major crash caused by AI to scare the capital owners.

All the previous human-driven crashes didn't change anything about capital owners' approach to money, so why would an AI-driven crash change things?

>>AlotOf+N2
Consider how many edge cases it misses. Equivalence probably shouldn't be the top priority here.

>>Dylan1+ap
Equivalence here would definitely be the worst test, except for all the alternatives.

>>onion2+in
The scapegoating is different. Using an LLM makes them more culpable for the failure, because they should have known better than to use a tech that is well known to systematically lie.

>>onion2+in
because we have an alternative that we humans can fix. The problem with AI is that it creates without leaving a trace of understanding.

>>simmer+(OP)
That will get less terrifying when they move on to medical.

>>AlotOf+N2
They don't care. This is clearly someone looking to score points and impress with the AI magic trick.

The best part is that they can say the AI will get some stuff wrong, they knew that, and it's not their fault when it breaks. Or more likely, it'll break in subtle ways, nobody will ever notice and the consequences won't be traced back to this. YOLO!

>>simmer+(OP)
I work for a huge bank: The actual terrifying thing is that these financial models semi-technical management are creating in Excel are actually relied upon in the first place.

If you convert bullshit from Excel to Python it's still bullshit. There's a reason why Claude can one-shot it and no one questions the result :D

>>nebula+03
Most of the large outages at Meta in the past 10 years were related to early AI automation.

>>simmer+(OP)
Someone recently had AI create a trading bot and it returns 131% on every transaction over a 30 day period - do you really think they care about code quality or ability to verify the math?

>>pizzaf+NA1
ANd I have a bridge to sell you in London

>>simmer+(OP)
Yep. And I think many are too focused on just the pure ability to build models now, when in reality a model is at its core a set of assumptions. Using models you don't understand, didn't build, and barely reviewed amplifies the magnitude of any assumption error, imo.

>>daniel+Xg
Well I did say "maybe". It all depends on how badly they get burned. A crash will hopefully scare enough of them from trying for a while.

>>riskab+cz1
The Python bullshit at least can be unit tested bit by bit and easily improved.

Nobody will see that on sheet 27 cell FG456 is actually a static number that Brian typoed in there in 2019 and not a formula.

>>simmer+Ak2
Someone bought that bridge and it stands in Arizona.