zlacker

>>Josely+(OP)
This is a bit off topic to the actual article, but I see a lot of top ranking comments complaining that ChatGPT has become lazy at coding. I wanted to make two observations:

1. Yes, GPT-4 Turbo is quantitatively getting lazier at coding. I benchmarked the last 2 updates to GPT-4 Turbo, and it got lazier each time.

2. For coding, asking GPT-4 Turbo to emit code changes as unified diffs causes a 3X reduction in lazy coding.

Here are some articles that discuss these topics in much more detail.

https://aider.chat/docs/unified-diffs.html

https://aider.chat/docs/benchmarks-0125.html

>>anothe+Pf
Can you say in one or two sentences what you mean by “lazy at coding” in this context?

>>omalle+6q
It has a tendency to do:

"// ... the rest of your code goes here"

in it's responses, rather than writing it all out.

>>Me1000+Mq
It's incredibly lazy. I've tried to coax it into returning the full code and it will claim to follow the instructions while regurgitating the same output you complained about. GPT-4 was great, GPT-4 Turbo first version was pretty terrible bordering on unusable, then they came out with the Turbo second version, which almost feels worse to me, though I haven't compared, but if someone comes claiming they fixed an issue, but you still see it, it will bias you to see it more.

Claude is doing much better in this area, local/open LLMs are getting quite good, it feels like OpenAI is not heading in a good direction here, and I hope they course correct.