zlacker

Well, the coding agent is pretty much a junior dev at the moment. The seniors are teaching it. Give it a 100k PRs with senior developer feedback and it'll improve just like you'd anticipate a junior would. There is no way that FANG aren't using the comments by the seniors as training data for their next version.

It's a long-term play to have pricey senior developers argue with an llm

>>baalim+(OP)
> using the comments by the seniors as training data for their next version

Yeah, I'm sure 100k comments with "Copilot, please look into this" and "The test cases are still failing" will massively improve these models.

>>baalim+(OP)
> Give it a 100k PRs with senior developer feedback

Don't you think it has already been trained with, I don't know, maybe millions of PRs?

>>baalim+(OP)
These things don't learn after training. There is no teaching going on here, and the arguments probably don't make for good training data without more refinement. That's why junior devs are still better than LLMs IMO, they do learn.

This is a performative waste of time

>>baalim+(OP)
A junior dev is (most often) a bright human being, with not much coding experience yet. They can certainly execute instructions and solve novel problems on their own, and they most certainly don't need 100k PRs to pick up new skills.

Equating LLMs to humans is pretty damn.. stupid. It's not even close (otherwise how come all the litany of office jobs that require far less reasoning than software development are not replaced?).

>>baalim+(OP)
at the very least, a junior shouldn't be adding new tests that fail. Will an LLM be able learn the social shame associated with that sort of lazy attitude? I imagine its fidelity isn't detailed enough to differentiate such a social failure from a request to improve a comment. Rather, it will propagate based on some coarse grained measures of success with high volume instead.

>>gf000+d3
A junior dev may also swap jobs, require vacation days, perks and can't be scaled up at a the click of a button. There are no such issues with an agent. So, if I were a FANG higher-up, I'd invest quite a bit into training LLM-agents who make pesky humans redundant.

Doing so has low risk, the senior devs may perhaps get fed up and quit, and the company might be a laughing stock on public PRs. But the potential value for is huge.

>>diggan+k1
Some of that seems somewhat strategic. With a junior you might do the same if you’re time pressured, or you might sidebar them in real life or they may come to you and you give more helpful advice.

Any senior dev at these organizations should know to some degree how LLMs work and in my opinion would to some degree, as a self protection mechanism, default to ambiguous vague comments like this. Some of the mentality is “if I have to look at it and solve it why don’t I go ahead and do it anyways vs having you do it” effort choices they’d do regardless of what is producing the PR. I think other parts of it is “why would I train my replacement, there’s no advantage for me here.”

>>baalim+(OP)
I’m curious why you think it hasn’t already been trained on 100ks or millions of PRs and their comments/feedback.

>>Frost1+R9
Sidebar? With a junior developer making these mistakes over and over again, they wouldn't even make it past the probationary period in their employment contract.

>>baalim+i5
It's probably easier to make the higher up redundant than to actually achieve high speed and predictable outcomes that satisfy real business needs and integrations in a cost effective way.

>>rchaud+xm
I guess it depends on how you view and interact with other people. I tend to give people the benefit of the doubt that they’re doing their best to succeed. Why wouldn’t you want to help them as much as you reasonably can, unless they’re actively a terrible person?

>>Frost1+Zu1
As a senior dev or manager, you're responsible for the people you've hired. Their mistakes become your mistakes. If they make the same kind of mistake repeatedly, and aren't able to take responsibility, you will have to clean up after them. They're not able to fulfill their job description and must be let go. That's why the probationary period exists.

Realistically, the issues occurring here are intern-level mistakes where you can take the time to train them, because expectations are low and they're usually not working on production-level software. In a FT position the stakes are higher so things like this get evaluated during the interview. If this were a real person, they wouldn't have gotten an offer at Microsoft.

>>baalim+i5
I mean, a Furby could respond to you all day, each hour, but that doesn't make them any more useful..

Not saying that LLMs are useless, but that's a false equivalency. Sure, my auto complete is also working 0-24, but I would rather visit my actual doctor who is only available in a very limited time frame.

>>baalim+(OP)
@Grok is this true?