Migrate thousands of tests to a new test framework
An LLM pipeline rewrites test files file by file, retries with error feedback, and leaves hard cases for engineers to finish.
The use case
Old test frameworks linger because rewriting thousands of files by hand is slow. Airbnb wrote about moving its React tests from Enzyme to React Testing Library with an LLM pipeline. Each file went through stages: migrate, then fix lint, type and test failures by feeding errors back to the model, with retries and richer context for stubborn files. Most files were done automatically and the remainder were finished by engineers. The pattern applies to any mechanical migration with a test suite as the check. Engineers still review migrated tests to make sure they test the same behavior, not just pass.
Example workflow
- Define the target pattern and collect example migrations
- Run each file through migrate, lint, typecheck and test stages
- Feed failures back to the model and retry with more context
- Send files that still fail to engineers
- Review a sample of passing files for weakened assertions
Tools
- LLM API
- Jest
- React Testing Library
Risks and guardrails
- Migrated tests can pass while asserting less than before
- Large batched PRs are hard to review; ship in small groups
- Track cost per file and stop runaway retries
Sources
Written by Agents Lookup. Credits and official pages for the tools this use case names:
- Airbnb engineering blog: test migration with LLMs: medium.com/airbnb-engineering/accelerating-large-scale-test-migration-with-llms-9565c208023b