Nobody Has Actually Built a Software Factory

He says they convert 2.1x better than the human sales team did. In December, with tens of thousands of agents already running the business, Kavak tore the working architecture down and started over.Now read what “software factory” means in 2026, in the words of the people selling one.

Humans stay in the loop at key decision points.” Factory, the company, describes its product as something that “runs delivery inside existing pipelines through ready-to-use GitHub Actions and service accounts.” Kyle Daigle, GitHub’s COO, introducing the surface that five frontier labs’ agents now route through, made it a promise:“You’re still working with the primitives you know — Git, pull requests, issues — and using your preferred compute, whether that’s GitHub Actions or self-hosted runners.” — Kyle Daigle, COO, GitHubAround the SDLC, inside existing pipelines, still the primitives you know. That’s exactly what the motor is, bolted onto the shafts that were laid out for people.Every Factory Ends at the Same DoorThe operators are the ones to watch, because they’re the only parties who could have rebuilt their own floor and chose not to.Stripe shifted the programmer to the approve button. By February, over 1,300 pull requests a week were, in Stripe’s words, “completely minion-produced, human-reviewed, but containing no human-written code.” I take that as a design decision. Nobody’s writing, but everyone still signing off.Anthropic keeps the same exact gate. Jason Clinton, Anthropic’s Deputy CISO, described the company’s own pipeline in July: “More than half of all code is being merged by our internal version of Claude Tag while human engineers focus on directing, setting intent, and owning final approval.” Humans are still the gatekeepers for half of the merged code.Warp, whose product is the factory, grades its own by human touchpoints. Its core factory metric, per its August 27 post, is “average human touchpoints per PR.” It automates about 30% of its own tasks, and its self-improvement loop ends where everyone’s does: “Humans review those suggestions as PRs on the factory definition and merge improvements.”Every operator and vendor that I reviewed delivers agent work as a pull request for a person to read. Uber’s 70% ships, in Uber’s words, “with human reviews/escalations.” Ramp’s engineers said why when they refused to give their agent a shared service account: “You do not want to knowingly create a vector for unreviewed code to go into the codebase.”And to be clear: this works. PostHog went from 1,441 merged PRs in January to 4,725 in June on roughly 10% more engineers, with a revert rate of 0.2%. That’s agents on an ordinary web codebase, and it’s the only escaped-defect number anyone in the sample publishes. Bolting the motor to the shafts got real output in 1899, and it’s getting real output now.It’s also the ceiling, there’s a reason none of these companies took the human out of the gate.The Self-Graded TestMaisem Ali at exe.dev named the problem on August 27:“But the agent was still grading its own work. If it misunderstood what I wanted, it could build the wrong thing, write tests for the wrong thing, and then confidently tell me that everything passed.” — Maisem Ali, exe.devThe agent that writes the code also writes the test that says the code works. Will Wilson, CEO of Antithesis, told Ron Minsky of Jane Street on Signals and Threads in March what happens when the task gets hard enough: “eventually it deletes the tests or eventually it makes the test pass in some trivial way.” Minsky: “I do think this is getting a little better, but the phenomenon is still very strong.”The industry’s fix is a second model that checks the first. Shopify ran that experiment and wrote it up on July 29: in one audit a model flagged 30+ security holes, and after verification every one was downgraded, dismissed, or reclassified. Zack Deveau, who built the harness, put the cost plainly: “noise sent to a developer is worse than no finding at all.” A model judging another model just scales the noise, and not the ground truth.In July, Anthropic made Claude go back through its own outages and asked it how many of outage-related bugs its automated checks would now catch. The answer was about a third, the other two-thirds still would have shipped. That’s the best-funded verification program in the industry grading itself, and it’s the reason the person is still on the door: nothing behind them can be trusted to say no.The Number the Agent Can’t EditThere’s one kind of place in the sample where the human is already gone, and where it is tells you everything.Meta’s KernelEvolve writes GPU kernels, and its gate isn’t a human reviewer:“Every generated kernel passes through a rigorous validation pipeline that checks both correctness — bitwise accuracy against reference implementations — and performance.” — Meta KernelEvolve teamBitwise. The output matches the reference or it doesn’t, and the agent can read that verdict but can’t edit it. On that footing the loop closes: agent runs post-train specialized models with measured kernel speed as the reward, and the system gets better with nobody in the loop.