research · 2026-09-11

Before an agent's code reaches production: a review checklist

Security pass rates for generated code have not moved in four years, duplication is up 81% and unreviewed merges up 31%. Five gates catch each of those before a person reads the code.

For founders and CTOs of SaaS companies and IT leaders who let coding agents write code that shipsyaab6 min read
Before an agent's code reaches production: a review checklist

This document describes the five gates that code written by an agent passes before it reaches production, in the order the code meets them, with the check to run at each and the published finding behind it. It ends with a five day plan to put the gates in place and how we run them on our own work.

1. What the published data says about generated code

Three sources, three different lenses, one direction.

Security. Veracode's 2026 GenAI Code Security Report tested 11 new models across 80 tasks and reports that across more than 100 models tested over four years, the average security pass rate sits at 56% and has not moved. The best model in the summer 2026 dataset passes 68% of security tasks; six of eleven sit between 50% and 53%. Java is the weakest language at a 30% mean pass rate. Models built for code average 51%; general purpose models, 52%. Being trained to write code faster does not mean writing it safer.

Maintainability. GitClear analyzed 623 million code changes from 2023 to 2026. Duplicated code blocks (five or more consecutive repeated lines) rose 81% since 2023 to the highest level on record. Moved code, the signature of refactoring, fell from 21% of changed lines in 2022 to 3.8% in 2026, while copy and paste climbed from 9.4% to 15.7%. New code connects to existing functions 35% less often. The share of changes that touch code older than a year fell 74%. Error masking constructs are up 47% and two week churn up 15%.

Flow. Faros AI's 2026 telemetry across 22,000 developers found pull requests merged with no review at all up 31.3% under high AI adoption, and incidents per merged pull request up 242.7%.

The checklist exists because every one of these is a measurable property of a change, and every one can be checked before a human sees it.

2. The principle: sensors before eyes

The harness engineering framework (Böckeler, martinfowler.com, April 2, 2026) gives the vocabulary. An outer harness around a coding agent has two kinds of controls. Guides act before the agent writes: conventions, constraints, instructions for a task. Sensors act after: tests, linters, structural checks, review instructions. Guides raise the chance the agent gets it right the first time; sensors let it correct itself before a person is involved.

Each control is either computational (deterministic, fast, run by the CPU: tests, type checks, linters, architecture tests) or inferential (a model judging code: semantic duplication, over engineering, a review against a rubric). Computational sensors are cheap enough to run on every change. Inferential ones are slower and probabilistic, and they catch what the deterministic ones cannot.

The framework's timing rule is the one that organizes the gates: keep quality left. Run what is fast before integration, run what is expensive after, and run drift sensors continuously against the codebase outside the change lifecycle. The human's job is the steering loop: every time an issue repeats, the harness gains a guide or a sensor so it repeats less.

3. The five gates

Gate 1. Before the agent writes.

Gate 2. While the agent works.

Gate 3. Before merge.

Gate 4. After merge.

Gate 5. The human's job.

4. Running it in a week

Day one: write the conventions file and the review rubric, both short. Day two: turn on the computational sensors that already exist in the stack (tests, types, linters) as agent facing checks, and add the duplicate block tripwire. Day three: add static security analysis to the pull request and set the size limit. Day four: write the review by exception list and remove the path that lets a change merge with no review. Day five: put the three post merge measures on a page someone looks at monthly. Gate 5 starts the first time something gets caught twice.

5. How we run it on our own work

On our own work, and inside the SaaS teams where we run it, a coding agent does not open a pull request until its own tests, the type checker, the linters and the structural test pass, and the review rubric runs before a person is assigned. The conventions file is the first thing written in a new repository and the first thing updated when a reviewer catches a repeat. The size limit is enforced by the pipeline, not by asking. Security analysis runs on the diff, and the language's weak spots are stated in the task. The post merge measures are read monthly and the harness changes when they move.

That is the method. The checklist is the version of it that fits on one page.

Sources

Analysis and conclusions by yaab, based on the published reports listed above.