zlacker

I tried the agent thing on:

- Large C codebase (new feature and bugfix)

- Small rust codebase (new feature)

- Brand new greenfield frontend for an in-spec and documented openAPI API

- Small fixes to an existing frontend

It failed _dramatically_ in all cases. Maybe I'm using this thing wrong but it is devin-level fail. Gets diffs wrong. Passes phantom arguments to tools. Screws up basic features. Pulls in hundreds of line changes on unrelated files to refactor. Refactors again and again, over itself, partially, so that the uncompleted boneyard of an old refactor sits in the codebase like a skeleton (those tokens are also sent up to the model).

It genuinely makes an insane, horrible, spaghetti MESS of the codebase. Any codebase. I expected it to be good at svelte and solidJS since those are popular javascript frameworks with lots of training data. Nope, it's bad. This was a few days ago, Claude 4. Seriously, seriously people what am I missing here with this agents thing. They are such gluttonous eaters of tokens that I'm beginning to think these agent posts are paid advertising.

>>mlsu+(OP)
How are you writing your prompts? I usually break a feature down to smaller task level before I prompt an agent (claude code in my case) to do anything. Feature level is often too hard to prompt and specify in enough detail for it to get right.

So I'd say claude 4 agents today are at smart but fresh intern level of autonomy. You still have to do the high level planning and task break down, but it can execute on tasks (say requiring 10 - 200 lines of code excluding tests). Any asking it to write much more code (200+ lines) often require a lot of follow ups and disappointment.

>>mlsu+(OP)
Have it make small changes. Restrict it to a single file and scope it to <50 lines or so. Enough that you can easily digest without making it a chore.

>>chuckn+d1
This is the thing that gets me about LLM usage. They can be amazing revolutionary tech and yes they can also be nearly impossible to use right. The claim that they are going to replace this or that is hampered by the fact that there is very real skill required (at best) or just won't work most the time (at worst). Yes there are examples of amazing things, but the majority of things seem bad.

>>mlsu+(OP)
It’s entirely possible that the people talking up agents also produced spaghetti code but don’t care because they are so much more “productive”.

An interesting thing about many of these types of posts is they never actually detail the tools they use and how they use them to achieve their results. It shouldn’t even be that hard for them to do, they could just have their agent do it for them.

>>mlsu+(OP)
>these agent posts are paid advertising.

I'm 100% certain most if not all of them are, there is simply too much money flying around and I've seen things that marketing does in the past for way less hyped products. Though in this specific case I think the writer may simply be shilling AI to create demand for their service. Pay us monthly to one click deploy your broken incomplete AI slop. The app doesn't work? No problem just keep prompting harder and paying us more to host/build/test/deploy it...

I've also tried the agent thing and still am with only moderate success. Cursor, Claud-squad, goose, dagger AI agents. In other words all the new hotness, all with various features claiming to solve the fact that agents don't work. Guess what? they still don't.

But hey this is HN? most of the posters are tech fearing luddies right? All the contention on here must mean our grindset is wrong and we are not prompting hard enough.

There is even one shill Ghuntly that claims you need to be "redlining" ai at the cost of $500-$1000 per day to get the full benefits. LOL if that is not a veiled advertisement I don't know what is.

>>mlsu+(OP)
This is my experience too most of the time. Though sometimes it does work, and sometimes a solution is found that I never thought of. But most of the time I have to change things around to my liking.

However, a counter argument to all this;

Does it matter if the code is messy?

None of this matters to the users and people who only know how to vibe code.

>>chuckn+d1
Coding agents should take you through a questionnaire before working. Break down what you are asking for into chunks, point me to key files that are important for this change, etc etc. I feel like a bit of extra prompting would help a lot of people get much better results rather than expecting people to know the arcane art of proompting just by looking at a chat input.

>>jvande+H1
I have not had a ton of success getting good results out of LLMs but this feels like a UX problem. If there’s an effective way to frame a prompt why don’t we get a guided form instead of a single chat box input?

>>turtle+B1
A small change scoped to <50 lines is something easy to write for a normal software engineer. When do the LLMs start doing the hard part?

>>CapsAd+G3
> Does it matter if the code is messy?

It matters proportionally to the amount of time I intend to maintain it for, and the amount of maintenance expected.

>>declan+W3
When you wire them up to your cicd process with pull requests and the github gui as your interface, rather than sitting there passively riding along as it prompts you the changes it’s going to make.

With my async agent I do not care about how easy it is for me, it’s easier to tell the agent to do the workflow and comeback to it later when I’m ready to review it. If it’s a good change I approve the pr, if not I close it.

>>mlsu+(OP)
You’re not providing a key piece of information to provide you with an answer: what were the prompts you used? You can share your sessions via URL.

A prompt like “Write a $x program that does $y” is generally going to produce some pretty poor code. You generally want to include a lot of details and desires in your prompt. And include something like “Ask clarifying questions until you can provide a good solution”.

A lot of the people who complain about poor code generation use poor prompting.

>>vitafl+C2
>> It’s entirely possible that the people talking up agents also produced spaghetti code but don’t care because they are so much more “productive”.

You may be right. The author of this one even says if you spend time prettying your code you should stop yak shaving. They apparently don't care about code quality.

>>presen+I3
I am just a muggle, but I have been using Windsurf for months and this is the only way for me to end up with working code.

A significant portion of my prompts are writing and reading from .md files, which plan and document the progress.

When I start a new feature, it begins with: We need to add a new feature X that does ABC, create a .md in /docs to plan this feature. Ask me questions to help scope the feature.

I then manually edit the feature-x.md file, and only then tell the tool to implement it.

Also, after any major change, I say: Add this to docs/current_app_understanding.md.

Every single chat starts with: Read docs/current_app_understanding.md to get up to speed.

The really cool side benefit here is that I end up with solid docs, which I admittedly would have never created in the past.

>>chuckn+d1
I feel like when you prompt an LLM the LLM should take it almost as "what would the best possible prompt for this prompt be and then do that"...

>>runjak+05
It'd be nice if the AI advocates shared prompts, or even recorded entire sessions. Then we could all see how great it really is.

>>phkahl+b5
Many do. In their GitHub repos right next to the code.

>>runjak+05
Prompt engineering isn't really that important anymore imo. If you're using a reasoning model, you can see if it understood your request by reading the reasoning trace.

>>vitafl+C2
The agent/model being used makes a huge difference. Cline with Claude 3.7 is ridiculously expensive but useful. Copilot is vaguely ok.

>>presen+I3
I mean if you ask Claude code to walk through what you should do next with you it'll ask lots of great questions and write you a great TODO.md file that it'll then walk down and check the boxes on.

You don't exactly need to know prompting, you just need to know how to ask the AI to help you prompt it.

>>declan+W3
A small change around 50 lines is the size of an advent of code solution (the hardest part). Most of the code you write around that is for defensive coding (error handling, malformed input, expected output,…) which is the other hard part. Then you connect these cores to form a system and that’s another tough problem. And it needs to evolve to.

We’ve built tools to help us with the first part, framework with the second, architecture principles with the third and software engineering techniques for the fourth. Where do LLMs help?

>>phkahl+b5
They do. All over the place.

Simon Willison has some great examples in his blog and on his GitHub. Check out Karpathy’s YouTube videos as well.

https://simonwillison.net/

>>phkahl+35
> You may be right. The author of this one even says if you spend time prettying your code you should stop yak shaving. They apparently don't care about code quality.

brought to you by fly.io, where the corporate blog literally tells you to shove your concerns up your ass:

> Cut me a little slack as I ask you to shove this concern up your ass.

>>phkahl+b5
This one's been on the HN front page all day:

>>44159166

>>lexand+m5
Yeah I strongly disagree with that. I think prompts are very critical.

As with any other project, it’s best to specify your wants and needs than to let someone or an LLM to guess.

>>mlsu+(OP)
I don't think it's fair to call that the agent thing. I've had profoundly positive results with agentic workflows for classification, analysis, and various business automations, including direct product pricing. You have to build an environment for the agent to make decisions in, with good instructions for what you want them to do. Then you wire it up so that the decisions have effects in the real world. You can acheieve really good results, and there is a lot of flexibility to tweak it and various tricks to optimize performance. Tools can allow agents to pull in relevant context as needed, or to execute complex multistep workflows. That is the agent thing.

Writing code is one thing that models can do when wired properly, and you can get a powerful productivity boost, but wielding the tools well is a skill of it's own, and results will vary by task, with each model having unique strengths. The most important skill is understanding the limitations.

Based on your task descriptions and the implied expectation, I'm unsurprised that you are frustrated with the results. For good results with anything requiring architecture decisions have a discussion with the model about architecture design, before diving in. Come up with a step by step plan and work through it together. Models are not like people, they know everything and nothing.

>>phkahl+b5
"Infrastructure as Prompts" (IaP)

>>lexand+m5
Disagree, effective prompting remains a crucial skill that's difficult to acquire.

I've been developing my prompting skills for nearly three years now and I still constantly find new and better ways to prompt.

I also consider knowing what "use a reasoning model" means to be part of that skill!

>>lumost+z5
Even just doing the cut and paste thing for one shots, claude sonnet 4 writes good rust code, generally on the first try.

>>citize+I2
Nobody has paid me to write anything about AI and if they did I would disclose it.

>>declan+W3
Can you type that 50 line change in less than 15 seconds?

>>presen+I3
You can ask it to do this, in your initial prompt encourage it to ask questions before implementing if it is unsure. Certain models like o4 seem to do this more by default rather than Claude that tends to try to do everything without clarifying

>>lexand+m5
That's a very dangerous thought. Prompt engineering evolved is just clear and direct communication. That's a hard thing to get right when talking to people. Heck, personally I can have a hard time with clear and coherent internal dialog. When I am working with models and encounter unexpected results, it often boils down to the model giving me what I asked for instead of what I want. I've never met anyone who always knows exactly what they want and is able to articulate it with perfect clarity. Some of the models are surprisingly good at figuring out intent, but complexity inevitably requires additional context. Whether you are working with a model or a person, or even your future self, you must spend time developing and articulating clear specifications, that is prompt engineering. Furthermore, models don't "think" like people--there's technique in how you struture specifications for optimal results.

>>runjak+05
There a many ways to do something wrong and few ways to do them right. It's on the AI advocates to show us session logs so we can all see how it's done right.

>>jacob0+28
Fair enough. I guess I was mainly thinking of how rarely I need to utilise the old prompt engineering techniques. Stuff like: "You are an expert software developer...", "you must do this or people will die." etc

I just tell the AI what I want, with sufficient context. Then, I check the reasoning trace to check it understood what I wanted. You need to be clear in your prompts, sure, but I don't really see it as "prompt engineering" any more.

>>mlsu+(OP)
Same here. I keep trying to figure out WTF agent that people are using to get these great results, because Copilot with Claude 4 and Gemini 2.5 has been a disastrous mess for me.

>>simonw+b7
ditto - https://ghuntley.com/disclosures

>>ghuntl+jf1
Forgive me if I don't consider your personal blog an authority of your honesty. "I'm not a liar, for real dude see it says right there"

Why are all your public projects "joke/toy projects" if AI is so awesome and production ready? My experience reflects this as the truth. Yet your work backs up my experience rather than your words.

To avoid only being snark. I think all software is about power/control and software has allowed unprecedented concentration of power Which is why it resists being official like other industries. No one with power wants restrictions on their power/software. Ultimately AI is good for small'ish projects and a productivity multiplier(eventually). I think it will lead a new revolution in distilling the current incumbents in the business world that are stagnant on vast proprietary software systems that previously could not be unseated. Small players will be able to codify/automate their business to make it competitive with big players. So I'm not "anti-ai".

edit: AI will simultaneously rot the proprietary software advantage from the inside out. As companies are further convinced that AI can solve their problem of having to pay people to maintain their software.

>>citize+w32
I think it's pretty counter-productive to default to not trusting anyone under any circumstances.

Having a "disclosures" page on a personal website is a pretty strong quality signal for me - it's inspired me to set a task to add my own.

As with all of these things, the trick is to establish credibility over time. I've been following Geoff for a few years. Given his previous work I think he has integrity and I'm ready to believe his disclosures page.

>>simonw+952
I get what your saying.

However we seem to live in a time where integrity is basically valued at zero or more commonly as something to "bank" so you can cash in for an enormous payoff when the time comes. I agree he seems authentic, therefore valuable. Which means an AI company can come and offer him 7-8 figures to build hype. I think its hard for people to truly grasp just how much money is flying around in hype cycles. Those numbers are not unrealistic. That's set for life money, not many are in a position to refuse that kind of wealth. (he lives in van, just saying)

I hope he is one of the few authentic people left but the world has left me jaded.

>>citize+hb2
Secretly offering someone 7-8 figures to hype for you is a big business risk to take on.

If details of that deal leak, it's a big embarrassment for the company.

In the USA it is also illegal. There are substantial FTC fines to worry about. If it affects the stock price it could be classified as securities fraud (Matt Levine will happily tell you that "everything is securities fraud").

>>simonw+i7
It'd take less than the time it takes to come up with The Perfect Prompt.

>>simonw+tE2
>If details of that deal leak, it's a big embarrassment for the company.

Intermediaries.

Also IMO the risk of someone whom is not already rich turning down that kinda money is so close to zero that it is effectively zero. No risk.

If everything is securities fraud then by that logic it would not be considered in making sketchy deals. Also as you double state, it only matters if the company is public anyway. Hmmm is openai public? Are any of the AI players besides MS,Oracle,Google? Short answer. No.

I'm not sure why with all the public unpunished criminal behavior we see nowadays you have such trouble believing that there really are lots of paid shills for such a hyped product.

>>ghuntl+jf1
So you have no reply to my straightforward question? If AI has made you 1000x productive why do you only post self described toy/joke projects?

I know the answer though. Its because that is all it is good for. Everyone that has tried these tools knows that.

>>citize+9y5
Probably because, as an unpaid shill, I understand why people would recommend this stuff without needing to be paid to do so.

>>simonw+nE5
I'm not accusing you of being a paid shill. My core argument is that there are lots of paid shills for AI being created for the last 2 years and going. I of course will never have the hard evidence to prove it so inferring is all I can do or point people to.

HN seems to have a very long tolerance of suspected/potential white collar crimes. So I don't expect many allies on here. Seems the mindset is the ends justifies the means prevails.