AI-assisted code review: what Google, Meta and GitHub have published, and a study that disagrees

Google trained a model to propose the code change that addresses a reviewer's comment. In offline evaluation it addressed 52% of comments at a target precision of 50%; in production, authors apply 40% to 50% of the suggested edits they preview ^[1]. The design choice worth copying: suggestions are shown only above a confidence threshold, and the author always decides.

Meta's TestGen-LLM generates additional unit tests for existing classes and keeps only those that build, pass reliably and increase coverage. In its evaluation, 75% of generated tests built correctly, 57% passed reliably and 25% increased coverage; engineers accepted 73% of the improvements it recommended ^[2]. The filters, not the model, are what make the output trustworthy.

GitHub's controlled experiment asked developers to write an HTTP server in JavaScript; those with Copilot finished 55% faster ^[3]. METR's 2025 randomised trial instead had experienced open-source developers work on real issues in repositories they knew well; with AI tools they took 19% longer, even though they expected to be faster and afterwards believed they had been ^[4].

The studies are not contradictory so much as about different work: a small new task versus changes to a large codebase the developer knows deeply. Google's DORA research similarly reports that AI adoption brings gains for individuals but warns about effects on delivery stability ^[5].

• Use AI where output can be verified automatically: tests that must pass, suggestions above a confidence threshold, lint and security findings.
• Measure delivery outcomes (lead time, change failure rate), not lines generated.
• Keep human review on design and on changes to critical code paths.
• Run your own trial on your codebase before rolling out widely.

Found this useful?

Share the key finding on X.