Managing AI Code Quality at Scale
Last updated . Sources are named and dated inline - how we source claims.
The engineering practice - what changes in a pull request, what a reviewer does differently, and which repository checks catch AI-generated defects before they merge
What This Page Covers
This page covers the day-to-day mechanics of AI-assisted code at the pull request and repository level: what a reviewer reads differently, which checks run before a human ever opens the diff, how large an AI-assisted change is allowed to get, how prompt habits change what lands in review, and which repository signals tell you the practice is working. It is what happens in a pull request, not what leadership signs off on.
For the organizational policy - approved tools, acceptable use, layered controls, policy templates, rollout phases, training programs, audit trails, and governance metrics - see the AI Coding Governance Framework. That is the canonical page for "we need an AI coding policy," and everything on this page assumes such a policy already exists.
For non-AI technical debt governance - debt classification, capacity allocation, the debt register, definitions of done, escalation, and reporting cadence - see Tech Debt Governance and Policy.
What Changes in the Diff
A reviewer cannot apply the same reading to AI-assisted code that they apply to a colleague's. The failure modes are different, they are systematic rather than idiosyncratic, and three of them are measurable in the diff itself. Everything later on this page is a response to one of these four.
Duplication Instead of Reuse
An assistant that cannot see the helper you already wrote will write it again. GitClear's analysis of 623 million changes from 2023 to 2026 found block duplication up 81%, copy-pasted lines rising from 9.4% of changed lines in 2022 to 15.7% in the first half of 2026, and moved code - the signature of refactoring - falling from 21% to 3.8%.
What the reviewer does: search the repository for the function before approving it. The duplicate is usually two directories away and passes every test.
Source: GitClear, The Maintainability Gap: AI Code Quality in 2026 (2026)
Security Flaws That Compile Cleanly
Veracode tested code generated by more than 100 large language models across Java, JavaScript, Python, and C# and found that AI-generated code introduced risky security flaws in 45% of tests. Veracode sells security tooling, so read the number with that interest in mind - but the failure classes it names are the ordinary ones: injection, weak crypto defaults, missing output encoding.
What the reviewer does: treat every generated boundary - input parsing, query construction, template rendering, deserialization - as unreviewed until proven otherwise.
Source: Veracode, 2025 GenAI Code Security Report (2025)
Imports for Packages That Do Not Exist
Researchers presenting at the 34th USENIX Security Symposium generated 576,000 code samples with 16 models and found 205,474 unique hallucinated package names, with hallucination rates of at least 5.2% for commercial models and 21.7% for open-source models. An attacker who registers a frequently hallucinated name owns your next install.
What the reviewer does: verify that every newly added dependency exists, is the package it claims to be, and has a plausible publisher and download history. This is a lockfile question, not a code question.
Source: Spracklen et al., 34th USENIX Security Symposium (2025)
No Author To Ask
Ordinary review leans on a conversation: why did you do it this way, what did you consider, what happens at the boundary. When the answer is "the assistant produced it and the tests passed," that conversation has no other end. The submitter may not be able to defend a line they did not write and did not read closely.
What the reviewer does: ask the submitter to explain the change in their own words before approving. If they cannot, the change is not ready, regardless of what the pipeline says.
The Review Gate: What a Reviewer Does Differently
Ordinary review asks "is this correct?" AI-assisted review has to also ask "is this ours, and did anyone actually read it?" These six checks are the difference. They are cheap individually and they catch the failure modes above before merge, which is the only point at which the fix is still small.
1. Author Attestation Before Review Starts
The pull request template carries one required box: the submitter confirms they have read every changed line and can explain it. This is not paperwork - it moves the first read from the reviewer back to the person who asked for the code. Reviewers are allowed to bounce an unattested pull request without reading it.
2. Does It Match This Codebase?
Generated code is fluent in the average of all codebases and native to none. Check the error handling convention, the logging call, the configuration lookup, the data access layer. Code that is idiomatic in general but foreign here is the most common thing a passing test suite will happily let through.
3. Search Before You Approve
For every new helper, utility, constant, or type in the diff, grep the repository for an existing one. Reviewers who skip this step are the reason duplication climbs. Where the tooling supports it, let the duplication detector post the candidate matches as a review comment so the search is already done.
4. Read the Tests Harder Than the Code
Generated tests tend to assert what the generated code does rather than what the requirement says, which makes them tautologies that go green forever. Check that at least one test would fail if the implementation were wrong: null and empty inputs, boundary values, the error path, and the concurrent case. A coverage number cannot tell you this - only reading can.
5. Verify Every New Dependency by Hand
Any diff that touches a manifest or lockfile gets a dependency review: does the package exist, is the name spelled the way the real package is spelled, who publishes it, how old is it, and is the license compatible? A new transitive dependency arriving with a one-line feature is a finding, not a detail.
6. Confirm the Change Is Only What Was Asked
Assistants are generous. A request to fix one function comes back with a renamed variable three files over, a reformatted block, a helpfully "improved" default, and a dependency bump. Everything outside the stated purpose of the pull request either gets removed or gets its own pull request. Unasked-for changes are where the unreviewed lines hide.
Checks That Run Before a Human Looks
Reviewer attention is the scarcest thing in this process, so nothing a machine can decide should reach a person. Every check below is mechanical, runs on the pull request, and either passes or blocks - none of them require judgement, and that is exactly why they belong in the pipeline rather than in the review conversation.
Pipeline Gates
- - Cyclomatic and cognitive complexity ceilings per function
- - Coverage floor on changed lines, not on the whole repository
- - Cross-repository duplication detection on the added blocks
- - Dependency existence and integrity validation at build time
- - Mutation testing on the modules with the highest blast radius
Lint Rules Worth Adding
- - Unused imports and unreachable branches, which arrive in bulk
- - Empty catch blocks and swallowed errors
- - Commented-out alternative implementations left in the diff
- - Naming and file layout conventions specific to this repository
- - Architecture boundary rules, so a layer cannot import across itself
Security Scans
- - Static analysis on every pull request, blocking on high severity
- - Software composition analysis against the changed lockfile
- - Secret detection, including plausible-looking placeholder keys
- - License compatibility for every newly added package
- - Diff-scoped results, so the report is about this change only
Scope every result to the diff. A scanner that reports the whole repository's findings on every pull request trains developers to ignore it within a week. Report only what this change introduced, block only on what this change introduced, and the signal survives.
Diff Size Is the Control Nobody Configures
Every other check on this page degrades as the diff grows, and generation has removed the natural brake on diff size: producing 900 lines now costs the same as producing 90. Review quality does not scale the same way, so the limit has to be set deliberately.
Set a Ceiling, Then Enforce It
Pick a changed-line ceiling your reviewers can actually hold in their heads, measure your own team's current median before choosing it, and have the pipeline warn above that and require an explicit split justification well above it. Generated code should not get a larger allowance than handwritten code; if anything it needs a smaller one, because less of it has been read.
One Intention Per Pull Request
Mechanical changes - formatting, renames, generated files, dependency bumps - travel in their own pull request, never bundled with behaviour changes. A reviewer who has to find eleven meaningful lines inside four hundred mechanical ones will find nine of them. Separating the two is the cheapest review-quality improvement available.
Budget the Review, Not Just the Work
If generation halves the time to produce a change, the saved time belongs to review, not to the next ticket. Teams that book the whole saving as throughput are converting the gain directly into unread lines. Track review time per changed line alongside delivery, so the trade is visible instead of silent.
Prompt Hygiene at the Keyboard
Most review findings are decided before review, at the moment the request is typed. These are habits for the developer, not rules for the organization - they cost nothing and they shrink the diff that a colleague eventually has to read.
Give It the Context You Would Give a New Hire
- - Point at the existing module that already does the similar thing
- - Name the conventions: error type, logging call, config source
- - State the constraints that are not in the code, such as latency budgets
- - Say which libraries are already in use so it does not invent a new one
Ask Small, Then Ask Again
- - One function or one behaviour per request, not one feature
- - Reject and re-ask rather than patching output you do not understand
- - Ask for the test first, then the implementation, so the test is not fitted to the code
- - Ask what the code does at the boundaries, then verify the answer yourself
Read It Before It Becomes a Commit
- - Never accept a suggestion you have not read end to end
- - Delete the parts you did not ask for before staging
- - Look up every unfamiliar API call rather than trusting the signature
- - Run it once against a real input, not only against the generated test
Share What Worked, In the Repository
- - Keep the prompts that produce conforming code next to the code
- - Version them with the repository so they age with the conventions
- - Record the ones that reliably fail, and what to do instead
- - Prefer a checked-in context file over each developer's private habits
Repository Signals Worth Tracking
These are repository and pull request signals, not program metrics - they tell a team whether its own review practice is holding. Resist the urge to import a target from an article, including this one: measure your own baseline first, then watch the direction of travel. A number with no local history behind it is a number nobody will act on.
| Signal | Where the Threshold Comes From | What It Tells You |
|---|---|---|
| Post-merge defect density | Your own pre-adoption rate for the same repositories | The single most direct answer to whether the review gate is working. Compare AI-assisted pull requests against the rest of the same team's work, not against another team. |
| Two-week code churn | Your trailing twelve-month median for the repository | Lines rewritten shortly after merge were merged before they were understood. Rising churn is the earliest visible symptom of review that has become a formality. |
| Duplicate block ratio | The ratio measured before assistants were introduced | Directly tests the failure mode in the first section. If it climbs while feature output climbs, reviewers are not searching before approving. |
| Review revisions per pull request | Your team's median across the last two quarters | A jump means submitters are shipping output they have not self-reviewed, and the attestation step is being clicked rather than done. |
| Changed lines per pull request | The ceiling your team agreed to and can defend | A rising median with flat review time is the arithmetic of unread code. Watch the two together or neither is meaningful. |
| New findings per merged change | Zero high severity, by policy, from the first day | This is the one signal with an absolute floor rather than a baseline. Anything high severity that reaches main means a gate is misconfigured, not that the target was ambitious. |
Frequently Asked Questions
Smaller than a handwritten one, because less of it has been read. Generation removed the natural brake on diff size - producing 900 lines now costs what 90 used to - while review capacity did not change. Measure your team's current median changed-line count, set a ceiling reviewers can genuinely hold in their heads, warn above it in the pipeline, and require an explicit justification well above it. Keep mechanical changes such as formatting, renames, and dependency bumps in separate pull requests from behaviour changes, so meaningful lines are never buried in noise.
Most of them are not human work at all - complexity ceilings, duplication detection, dependency validation, and security scanning run in the pipeline, where they cost build minutes rather than reviewer attention. The human steps are deliberately short: an attestation checkbox, a repository search before approving a new helper, and a read of the tests. What actually costs time is rework on code that merged unread, and that cost is real whether or not anyone is measuring it. Track review time per changed line and defect escape rate across the rollout so the trade-off is measured rather than argued.
Six signals, each compared against your own baseline rather than an external benchmark: post-merge defect density for AI-assisted pull requests versus the same team's other work, two-week code churn, duplicate block ratio, review revisions per pull request, changed lines per pull request watched alongside review time, and new security findings per merged change. Only the last has an absolute target - zero high severity reaching main, because anything else means a gate is misconfigured. Present all of them as trends, never as a leaderboard; the moment these become a ranking, teams optimize the number instead of the practice.
Related Resources
AI Governance Framework
The organizational policy behind these practices: approved tools, layered controls, rollout phases, and governance metrics.
AI Code Review Guide
Line-level review techniques for evaluating AI-generated code once it is in front of you.
Copilot Anti-Patterns
The recurring shapes of AI-assisted code that look fine in the editor and cost you later.
Practice Needs Policy Behind It
Review gates hold only when the organization has agreed what they are for. Start with the policy your leadership signs off on, or measure the debt your current practice is already producing.