Extend existing unit tests, keeping only ones that add coverage
An agent proposes extra test cases and a filter keeps only those that build, pass reliably and raise coverage.
The use case
Coverage gaps are easy to find but tedious to fill. Meta's TestGen-LLM work and the open-source Qodo Cover tool both add new tests to existing test files, then throw away any candidate that fails to build, fails, flakes across repeated runs, or does not increase coverage. What survives is offered to engineers as a change to review. Meta reported that engineers accepted most of the recommendations that reached them. Engineers still read each test to check it asserts real behavior, since a test can raise coverage while checking nothing useful.
Example workflow
- Pick a source file and its existing test file
- Generate candidate test cases
- Discard candidates that fail to build, fail or flake
- Keep only tests that increase coverage
- Engineer reviews the remaining tests before merging
Tools
- Qodo Cover
- LLM API
- Coverage tools
Risks and guardrails
- Coverage-raising tests can have weak or no assertions
- Tests can lock in current buggy behavior
- Run repeatedly to catch flaky tests
Sources
Written by Agents Lookup. Credits and official pages for the tools this use case names:
- Meta paper: TestGen-LLM (arXiv): arxiv.org/abs/2402.09171
- Qodo Cover on GitHub: github.com/qodo-ai/qodo-cover