The Diff Does Not Contain the Consequences


A reviewer opens a pull request and sees what changed. Files, lines, additions in green.

What they cannot see is what happens next. Which services call this. Whether a mobile client parses that field strictly. Whether an overnight batch job reads the column being renamed. Whether the service this touches had an incident last week.

None of that is in the diff, because a diff is a description of an edit rather than a description of an effect. We have built our entire quality gate around reading edits and then act surprised when consequences arrive later.

Review capacity stopped scaling this year

This has always been true. What changed is the volume.

Meta's RADAR paper from May 2026 reports lines of code per developer up 105.9% year over year, with agentic AI driving over 80% of that growth. In the same period, the proportion of diffs receiving timely review declined, producing backlogs of thousands of pending reviews on large teams.

Output roughly doubled. The number of humans available to read it did not.

The response most teams have reached for is more review. More reviewers, stricter policies, longer checklists. That does not work arithmetically, and there is a good argument it does not work qualitatively either. Anthropic's own April 2026 postmortem described regressions that got through multiple human and automated reviews, unit tests, end to end tests, automated verification and dogfooding, as noted in one 2026 roundup. Every gate fired. The thing still shipped.

Adding a seventh gate to a system where six did not catch it is not a plan.

The variable we sort by is the wrong one

Look at how review attention actually gets allocated. Diff size. Author seniority. Which team owns the repository. Sometimes a line count threshold.

Every one of those is a property of the change. None is a property of the consequence.

A one line CSS tweak in a shared file is low risk. A three line change to shared authentication middleware is not, and both are three lines. Sorting by size treats them identically, which means you are spending scarce attention roughly at random with respect to the thing you care about.

The framing that has been converging through 2026 is to route review by blast radius and reversibility instead. Changes to public contracts and core invariants get a human gate. Additive changes go through asynchronously. Internal reversible refactors need no human gate at all.

Addy Osmani puts the same idea more bluntly: set the dial by blast radius, not by guilt. Stop reviewing everything to the same depth, and spend human attention where being wrong is expensive.

I agree with all of that. It also raises an obvious problem, which nobody in these pieces quite says out loud.

Blast radius is not in the repository

To route by blast radius you have to know the blast radius. And that information does not live in the pull request.

The same local edit produces completely different downstream outcomes depending on who consumes it. A tolerant consumer ignores a new field. A strict mobile client crashes on it. An admin build fails because an interface no longer matches. Dashboards go stale, batch jobs fail overnight, webhooks drift.

Establishing which of those applies means knowing what calls the changed code, across repositories rather than within one. Which services depend on the service being changed. Which infrastructure runs it. What the rollback path looks like. Whether that service is currently healthy or has been paging people.

That is a set of relationship questions, and a diff has no relationships in it. It has text.

Which is why "assess the blast radius" has stayed a nice principle rather than a practice. Everybody agrees with it. Almost nobody can do it in the ninety seconds a reviewer actually has.

What would make it practical

The useful form of this is not a report generated afterwards. It is context available at review time, while the change is still cheap to adjust.

Concretely, the inputs worth having are unglamorous: is this touching production, is it a stateful resource, does it change identity or network configuration, how hard is the rollback, does the affected service have recent incidents, and does it have an owner.

This is where I think Vorr AI fits into a workflow that is otherwise entirely about code. It holds the relationships between code, services, infrastructure and deployments, so the questions that determine risk are answerable rather than aspirational. What calls this function across every connected repository. What depends on this service. What runs it, and in which environment. What deployed recently and what happened afterwards.

None of that is a judgement about whether the change is good. It is the context a reviewer needs before their judgement means anything, and at the moment most reviewers are forming that judgement from the diff alone plus whatever they happen to remember about the system.

The distinction I would hold on to is that a reviewer with the diff is reviewing an edit. A reviewer with the diff and the dependency picture is reviewing a change.

The second-order problem

There is a reason this is getting worse rather than staying constant, and it is not only volume.

GitClear's analysis across 211 million lines of code from 2020 to 2024 found refactored code dropping from 24.1% to 9.5%, with copy-paste exceeding refactoring for the first time on record.

Duplication is directly a blast radius problem. When the same logic exists in five places rather than one, changing it correctly means finding all five, and the reviewer looking at one of them has no signal that the other four exist. The thing that makes impact hard to assess is increasing at the same time as the volume of changes to assess.

What I would actually change

Stop tiering by author. Seniority is a proxy for care, not for consequence. A careful engineer changing shared auth middleware is a higher risk change than a careless one changing a log message.

Make reversibility an explicit field. Whether a change can be rolled back cleanly is one of the strongest available risk signals and almost nobody records it. A schema migration and a copy tweak should not enter the same queue.

Put the dependency picture in front of the reviewer, not in a weekly governance report. By the time a report exists the change has shipped.

And accept that most changes do not need a human. Meta's automated path landed over 331,000 diffs while keeping production incident rates at a fraction of the non-automated set. The gain there is not really automation. It is that human attention got concentrated on the changes where being wrong was expensive.

The uncomfortable version of all this is that we have spent years improving how carefully people read diffs, and the thing that actually determines whether a change breaks production was never in the diff.


Author note

What made this click was noticing how I review. I read the change, I decide whether the code is good, and I approve.

At no point in that process do I ask what depends on it, because finding out would take longer than the review itself and nobody expects it of me. So I have been assessing correctness and calling it risk assessment, which are not the same activity at all.

Mohan Gopi, Associate DevOps Engineer at Frigga Cloud, working on Vorr AI across AWS, GCP and Azure. LinkedIn.

Post a Comment

Previous Post Next Post