And then that becomes a bit of a barrier to contribution -- it's a tradeoff
I often write an entirely new test harnesses for a feature, e.g. for https://www.oilshell.org, particularly using other implementations as "oracles". All of these run in the CI - https://www.oilshell.org/release/latest/quality.html
The good thing is that it definitely helps me accept PRs faster. Current contributors are good at this kind of exhaustive testing, but many PRs aren't
Even so, I do find myself simply adding test coverage from time-to-time -- that's an important job of a maintainer