Human on the Loop

Last year METR ran a randomized controlled trial that should have ended the argument about AI coding productivity. Sixteen experienced open-source developers, 246 real tasks, projects they’d worked on for an average of five years. The tasks where AI was allowed took 19% longer.

What stayed with me is what they believed. Beforehand they forecast a 24% speedup. Afterward, having done the work, they still estimated 20%. Wrong by nearly forty points about something they had just lived through.

METR now thinks developers are more sped up in 2026 than those numbers suggest. But the same shape shows up wherever you measure delivery instead of asking people. Google’s DORA research estimated a 7.2% drop in delivery stability for every 25% increase in AI adoption. A year later, with 90% of nearly 5,000 professionals using AI, throughput had turned positive and stability had not.

AI made producing change cheaper. It did not make absorbing change cheaper.

The Bottleneck Moved and Nobody Moved the Org Chart

The clearest evidence comes from Google. In an experience report on AI-driven code migrations, 80% of the code in landed changes was fully AI-authored and the engineers estimated migration time fell by half. Then this, from a second migration in the same paper:

Google, with Google’s tooling and Google’s reviewers, throttling the machine because the humans couldn’t keep up. Sundar Pichai said in April that 75% of new code at Google is AI-generated. The load-bearing words are the four that follow: “and approved by engineers.”

Generation is nearly free. Verification is not. The tell is a review queue that doesn’t empty no matter who you move onto it. Speed up one side and leave the other where it was and you have a production-line problem. Production lines have a hundred years of answers.

You Cannot Check Your Own Work

The obvious fix is having the AI review the AI. Google DeepMind studied that and found models “struggle to self-correct their responses without external feedback, and at times, their performance even degrades after self-correction.” The word doing the work is external. Ask an agent to introspect and it agrees with itself, because it’s rerunning the reasoning that produced the output. Split that same model into separate roles that check each other and quality measurably improves, as researchers have shown with the weights held constant.

None of which is really about AI. Your accountant doesn’t audit himself. Separation of duties is old technology. We just hadn’t needed it for software agents.

So We Built a Company, Not an Agent

Five seats, and the figure above has them. Two separations do the work: the CTO reviews the spec before any code exists and the code again before it merges, and the QA manager tests the running software without having seen it built.

All five are AI agents, each holding its own identity: its own email account, its own account on the task board, its own account in the code repository. (We’ve written before about what goes into building one.) That sounds like bookkeeping. It isn’t. A separation an agent could decide to skip is not a separation. Enforced by branch protection across genuinely different accounts, it is.

Where the gates live matters more than what they check. A task board can’t refuse a transition, so we don’t pretend it can. The real gates sit in version control at the one irreversible seam: protected trunk, required checks, merge allow-listed to an identity that didn’t write the code.

An agent skips any step that isn’t mechanically required. So we stopped writing norms and started writing gates.

What That Looks Like on Two Real Tickets

Here’s an actual feature. When a second article restates a fact we already have, stop creating a duplicate entry and attach it as a corroborating source. I said that much and the product manager wrote it up with acceptance criteria.

The CTO agent failed it. Not the code, the spec: the criteria described a data model our schema doesn’t have and shouldn’t grow. It went back before a line was written, and kicked one decision to me: keep the simpler append-and-supersede model, or build real versioning. I answered in one message.

The rewritten spec passed. The engineer implemented it in forty minutes. QA failed it and kicked it back. The engineer fixed it, QA passed, the CTO merged, operations deployed and verified. Ten days intake to live, of which the coding was under an hour and my share was one decision.

Ten days sounds slow. It isn’t, and the reason matters: this was a low-priority ticket, and higher-priority work kept arriving and getting worked first. That’s exactly what you want a real engineering team to do. Clear what’s urgent, come back to the small stuff in the gaps.

The forty minutes isn’t the impressive part. Anyone with a chat window can get forty minutes of code. The impressive part is that the spec was rejected before it cost anything, by something structurally incapable of being the thing that wrote it.

Then it shipped with a hole in it.

The next day, verifying that same feature in production, the operations agent found the corroboration logic never reached the sources that needed it. 594 duplicate rows had quietly accumulated. QA had passed because it tested the functions, not the whole pipeline against realistic data. So it filed a bug against our own shipped work. That’s the ticket in the video.

Two things on it matter most. Before a line of code existed, the product manager found a way to satisfy every draft criterion by suppressing all knowledge-item creation, and added a criterion forbidding exactly that. And when the fix deployed, the operations agent refused to close on the absence of failure. It wanted “the actual proof this fix works, not just that it didn’t regress anything,” so it waited for the next morning’s run, watched nineteen duplicates absorbed correctly, and closed on that.

Human on the Loop

The distinction isn’t ours. It comes from US defense doctrine, and it classifies by where the human sits rather than by how good the machine is.

I hold three decisions: what to build, what to spend, and changes to the platform. I don’t review pull requests, approve deploys, or serve as QA. When something genuinely ambiguous appears it lands in a queue with my name on it. That’s the system working.

There’s a cost, and Lisanne Bainbridge named it in 1983 in “Ironies of Automation.” Automation converts the job into monitoring, people are bad at watching a system where almost nothing happens, and skills decay because the automation is reliable. You don’t dodge that by trying harder, you dodge it by making the vigilance mechanical. I am not the safety net. The gates are.

One episode reassures me more than any metric. During a rollback demonstration the CTO agent declined to authorize a destructive migration against live production data, judged the risk unjustified, and escalated to me rather than deciding alone. That meant revising a ruling I’d made that morning. I took its recommendation.

I’ll take an organization that argues with me over one that agrees with me.

Why This Changes the Economics

Nobody has credibly measured cost per feature for an agent team against a human team, and almost every number out there is marketing. So my claim is structural.

The expensive part of software was never typing. It was everything around typing: the spec meetings, the handoffs, the review queue, the days a ticket waits on someone with the right permissions. Rigor was expensive because a reviewer is a salary and a calendar, which is why it gets thinned first under pressure. In an agent organization the waiting is a setting and the second reviewer is a compute bill.

The Four Rules

1. Make it explicit. Anything an agent has to infer, it will infer differently next time.
2. Make it a gate. An agent skips any step that isn’t mechanically required. Norms are decoration.
3. Make it continuous. Work is pulled, not pushed, so nothing waits on a person to remember.
4. The verifier is never the writer. The structural rule the other three rest on.

What I’d Ask a CEO

The companies getting this wrong aren’t being lazy. They did the obviously reasonable thing, made one worker faster, and left the organization around that worker alone. None of this needs a frontier lab. It needs treating your development process as a system you design, not a habit you inherited.

So the question isn’t whether your engineers should use AI. That’s settled. It’s this: if your team could produce four times as many changes starting tomorrow, what in your process breaks first?

Whatever you just thought of is the actual project.