zlacker

>>Josely+(OP)
This is a bit off topic to the actual article, but I see a lot of top ranking comments complaining that ChatGPT has become lazy at coding. I wanted to make two observations:

1. Yes, GPT-4 Turbo is quantitatively getting lazier at coding. I benchmarked the last 2 updates to GPT-4 Turbo, and it got lazier each time.

2. For coding, asking GPT-4 Turbo to emit code changes as unified diffs causes a 3X reduction in lazy coding.

Here are some articles that discuss these topics in much more detail.

https://aider.chat/docs/unified-diffs.html

https://aider.chat/docs/benchmarks-0125.html

>>anothe+Pf
How is laziness programmatically defined or used as a benchmark

>>th0ma5+Kq
Personally I have seen it saying stuff like:

public someComplexLogic() { // Complex logic goes here }

or another example when the code is long (ex: asking it to create a vue component) is that it will just add a comment saying the rest of the code goes here.

So you could test for it by asking it to create long/complex code and then running the output against unit tests that you created.

>>makest+2s
Yeah this is a typical issue:

- Can you do XXX (something complex) ?

- Yes of course, to do XXX, you need to implement XXX, and then you are good, here is how you can do:

int main(int argc, char **argv) {

  /* add your implementation here */

}