zlacker

>>orph+(OP)
How well can copilot write unit tests? This seems like an area where it could be really useful and actually improve software development practices.

>>Spinna+ad
Writing tests for the sake of coverage is already practically useless which is what a lot of orgs do, This could maybe generate such tests. However it doesn't materially impact quality now, so not much difference if automated.

One of the main value props for writing meaningful unit tests, is it helps the developer think differently about the code he is writing tests for, and that improves quality of the code composition.

>>manque+Qe
Why is that useless? Codebases I have worked on that had high code coverage requirements had very little bugs.

* It promotes actually looking at the code before considering it done

* It promotes refactoring

* It helps to prevent breaking changes for stuff that wasn't supposed to change

>>Graffu+Vl
The example i usually give [1] in javascript is say you have function

    (x,y) => x + y

Orgs targeting code coverage write a test for 1,2 => 3 and get 100% coverage and then stop as there is no incentive to go further. They don't write tests for say

   (1,null)
   (null,null)
   ('x',1) 
   (NaN, Infinity) 

and so on.. these additional tests will improve coverage of scenarios and code coverage will not move.

I have seen projects where a test will have sequence of steps which trigger the code but the assertion is effectively true === true, or they will replicate the same function in the test instead of generating proper mock data or myriad different absurd testing approaches. This comes from twin pressure of showing coverage and having tests pass.

Coverage is also a challenge in code which uses AI/ML libraries or that use third party services. These really need statistical testing with large volume of diverse well maintained samples and results need statistical analysis for error rates not different from how manufacturing does it, I don't see that often. For code using face detection for example a single face getting detected or not is hardly an adequate test.

Finally, it is easier to improve coverage by testing simpler code than improve coverage/ refactor a function which has say 10 nested branches, so it is not uncommon to see 90% coverage and 10% of the most used / most error prone code poorly or not tested at all.

There are some methods to address these like mutation testing, do retros for failure of tests to capture production bugs, it is not easy to measure and coverage driven orgs will not see their metrics moving by doing these.

Well written test suites will also have good coverage, but not necessarily other way around. Developers who care and understand what they are doing and why they are doing it, will use coverage as only the first step to see where there are gaps in their tests.

Tests are also code that need peer reviewed and maintained, if the tests depend on implementation and constantly break or contains improper mocks or assert poorly. A lot of not well written tests is hindrance to development than aid it.

[1] Yes, most of these are not applicable in a strongly typed language, but it is far easier as a illustration .

>>manque+cq
I don't think there's Infinity in my language, what do you use it for except maths?