zlacker

>>Josely+(OP)
This is a bit off topic to the actual article, but I see a lot of top ranking comments complaining that ChatGPT has become lazy at coding. I wanted to make two observations:

1. Yes, GPT-4 Turbo is quantitatively getting lazier at coding. I benchmarked the last 2 updates to GPT-4 Turbo, and it got lazier each time.

2. For coding, asking GPT-4 Turbo to emit code changes as unified diffs causes a 3X reduction in lazy coding.

Here are some articles that discuss these topics in much more detail.

https://aider.chat/docs/unified-diffs.html

https://aider.chat/docs/benchmarks-0125.html

>>anothe+Pf
Lazy coding is a feature not a bug. My guess is that it breaks aider automation, but by analyzing the AST that wouldn't be a problem. My experience with lazy coding, is it omits the irrelevant code, and focuses on the relevant part. That's good!

As a side note, i wrote a very simple small program to analyze Rust syntax, and single out functions and methods using the syn crate [1]. My purpose was exactly to make it ignore lazy-coded functions.

[1]https://github.com/pramatias/replacefn/tree/master/src

>>empora+jN
It sounds like you've been extremely lucky and only had GPT "omit the irrelevant code". That has not been my experience working intensively on this problem and evaluating numerous solutions through quantitative benchmarking. For example, GPT will do things like write a class with all the methods as simply stubs with comments describing their function.

Your link appears to be ~100 lines of code that use rust's syntax parser to search rust source code for a function with a given name and count the number of AST tokens it contains.

Your intuitions are correct, there are lots of ways that an AST can be useful for an AI coding tool. Aider makes extensive use of tree-sitter, in order to parse the ASTs of a ~dozen different languages [0].

But an AST parser seems unlikely to solve the problem of GPT being lazy and not writing the code you need.

[0] https://aider.chat/docs/repomap.html

>>anothe+N31
>For example, GPT will do things like write a class with all the methods as simply stubs with comments describing their function.

The tool needs a way to guide it to be more effective. It is not exactly trivial to get good results. I have been using GPT for 3.5 years and the problem you describe never happens to me. I could share with you just from last week, 500 to 1000 prompts i used to generate code, but the prompts i used to write the replacefn, can be found here [1]. Maybe there are some tips that could help.

[1] https://chat.openai.com/share/e0d2ab50-6a6b-4ee9-963a-066e18...

>>empora+o81
The chat transcript you linked is full of GPT being lazy and writing "todo" comments instead of providing all the code:

  // Handle struct-specific logic here
  // Add more details about the struct if needed
  // Handle other item types if needed
  ...etc...
  

It took >200 back-and-forth messages with ChatGPT to get it to ultimately write 84 lines of code? Sounds lazy to me.