Human on the Loop

Last year METR ran a randomized controlled trial that should have ended the argument about AI coding productivity. Sixteen experienced open-source developers, 246 real tasks, projects they had worked on for an average of five years. The tasks where AI was allowed took 19% longer.

What stayed with me is what they believed. Beforehand they forecast a 24% speedup. Afterward, having done the work, they still estimated 20%. Wrong by nearly forty points about something they had just lived through.

METR now thinks developers are more sped up in 2026 than those numbers suggest. But the same shape shows up wherever you measure delivery instead of asking people. Google’s DORA research estimated a 7.2% reduction in delivery stability for every 25% increase in AI adoption. A year later, with 90% of nearly 5,000 professionals using AI at work, throughput had turned positive and stability had not.

AI made producing change cheaper. It did not make absorbing change cheaper.

The Bottleneck Moved and Nobody Moved the Org Chart

The clearest evidence comes from Google. In an experience report on AI-driven code migrations, 80% of the code in landed changes was fully AI-authored and the engineers estimated migration time fell by half. Then this, from a second migration in the same paper:

Google, with Google’s tooling and Google’s reviewers, throttling the machine because the humans couldn’t keep up. Sundar Pichai said in April that 75% of new code at Google is AI-generated. The load-bearing words are the four that follow: “and approved by engineers.”

Generation is nearly free. Verification is not. The tell is a review queue that doesn’t empty no matter who you move onto it. Speed up one side of the equation, leave the other where it was, and you have a production-line problem. Production lines have a hundred years of answers.

You Cannot Check Your Own Work

The obvious fix is to have the AI review the AI. Google DeepMind studied that and found models “struggle to self-correct their responses without external feedback, and at times, their performance even degrades after self-correction.” The word doing the work is external. Give an agent a failing test and it corrects fine. Ask it to introspect and it agrees with itself, because it’s rerunning the reasoning that produced the output.

Split that same model into an analyst, a coder, and a tester, though, and researchers at Peking University found the pass rate on the standard benchmark went from 57.3% to 74.4%. Same weights, different org chart. Old research on a saturated benchmark, but the number was never the point. Your accountant doesn’t audit himself. The person who requests a wire isn’t the person who releases it. Separation of duties is old technology. We just hadn’t needed it for software agents.

So We Built a Company, Not an Agent

Five seats. A product manager turns intent into a spec with testable acceptance criteria. A CTO reviews that spec before any code exists, then reviews the code before it merges. An engineer implements. A QA manager tests the running software against the criteria without having seen it built. An operations manager deploys and verifies in production.

All five are AI agents, and each holds its own identity: its own email account, its own account on the task board, its own account in the code repository. (We’ve written before about what goes into building one.) That sounds like bookkeeping. It isn’t. Separation of duties enforced by a policy an agent could skip isn’t separation of duties. Enforced by branch protection across different accounts, it is. The author of a change cannot approve it, because no mechanism exists to let him.

Where the gates live matters more than what they check. A task board can’t refuse a transition, so we don’t pretend it can. The real gates sit in version control at the one irreversible seam: protected trunk, required checks, merge allow-listed to an identity that didn’t write the code.

An agent skips any step that isn’t mechanically required. So we stopped writing norms and started writing gates. We didn’t design most of them up front. We added them after something got past us.

Walkthrough · 5 min Watch a ticket move through the line, seat by seat

What That Looks Like on One Real Ticket

Here’s an actual feature. When a second article restates a fact we already have, stop creating a duplicate entry and attach it as a corroborating source. I said that much, and the product manager agent wrote it up with acceptance criteria.

The CTO agent failed it. Not the code, the spec: the criteria described a data model our schema doesn’t have and shouldn’t grow. The ticket went back before a line was written. That kicked one decision to me: keep the simpler append-and-supersede model, or build real versioning. I answered in one message.

The rewritten spec passed. The engineer implemented it in forty minutes. QA failed it and kicked it back. The engineer fixed it, QA passed, the CTO reviewed and merged, operations deployed and verified. Ten days intake to live, of which the coding was under an hour and my share was one ruling.

The forty minutes isn’t the impressive part. Anyone with a chat window can get forty minutes of code. The impressive part is that the spec was rejected before it cost anything, by something structurally incapable of being the thing that wrote it.

Ten days, and almost all of it was queue. Each agent sweeps for what’s assigned to it on a cadence we’ve kept deliberately slow while the gates prove out, so the waiting is a config value. In a human organization it’s a person’s calendar. Only one of those changes on a Tuesday.

Human on the Loop

The distinction isn’t ours. It comes from US defense doctrine: a human in the loop authorizes each action, a human on the loop supervises and can halt, a human out of the loop isn’t there at all. It classifies by where the human sits, not by how good the machine is.

I hold three decisions: what to build, what to spend, and changes to the platform itself. I don’t review pull requests, approve deploys, or serve as QA. When something genuinely ambiguous appears it lands in a queue with my name on it. That’s the system working.

There’s a cost, and Lisanne Bainbridge named it in 1983 in “Ironies of Automation.” Automation converts the job into monitoring, people are bad at holding attention on a system where almost nothing happens, and skills decay because the automation is reliable. You don’t dodge that by trying harder. You dodge it by making the vigilance mechanical. I am not the safety net. The tests, the scans, the independent QA pass and the merge allow-list are.

One episode reassures me more than any metric. During a rollback demonstration the CTO agent declined to authorize a destructive migration against live production data, judged the risk unjustified, and escalated to me rather than deciding alone. That meant revising a ruling I’d made that morning. I took its recommendation.

I’ll take an organization that argues with me over one that agrees with me.

Why This Changes the Economics

Nobody has credibly measured cost per feature for an agent team against a human team, and almost every number out there is marketing. So my claim is structural.

The expensive part of software was never typing. It was everything around typing: the spec meetings, the handoffs, the review queue, the days a ticket waits on someone with the right permissions. And rigor was expensive because a reviewer is a salary and a calendar, which is why review gets thinned first under pressure. In an agent organization the waiting is a setting and the second reviewer is a compute bill.

Not “AI writes the code.” AI has written code for years. What changed is that rigor got cheap.

The Four Rules

1. Make it explicit. Anything an agent has to infer, it will infer differently next time.
2. Make it a gate. An agent skips any step that isn’t mechanically required. Norms are decoration.
3. Make it continuous. Work is pulled, not pushed, so nothing waits on a person to remember.
4. The verifier is never the writer. The structural rule the other three rest on.

None of this needs a frontier lab. It needs deciding that your development process is a system you design, not a habit you inherited.

What I’d Ask a CEO

The companies getting this wrong aren’t being lazy about it. They did the obviously reasonable thing, made one worker faster, and left the organization around that worker alone.

So the question isn’t whether your engineers should use AI. That’s settled. It’s this: if your team could produce four times as many changes starting tomorrow, what in your process breaks first?

Whatever you just thought of is the actual project.