One engineer with a phone can now outpace an entire team's review capacity
How to Build an AI Software Factory: Agents That Open, Review, and Merge PRs
An AI software factory is the infrastructure around a coding agent, not the agent itself. It has five stages—intake, isolation, tools, verification, and merge gate—each with a gate that stops work from moving forward. Companies like Stripe, Spotify, and Shopify built the gates before scaling the fleet. The core problem is that generation scales with spend while review does not, so the design must absorb agent output without drowning reviewers.
AI changes the economics of code production. One person with good judgment and a phone can generate PRs faster than a team can review them.
- Incipient
I'm using fable and opus about 25%75/% respectively. I have a 3 step process - discuss a topic and create a review, convert the review to a task file and update the design docs, then finally implement the task file in code. Normally around 300-1000 loc implemented for a change, more for a new UI page or whatever.
And honestly it's incredibly good.
- 50% of the time it'll generate code that I accept wholesale.
- 25%: it maybe makes a choice I don't like, or the implementation isn't the approach I liked, so I tweak it.
-10%: it needlessly creates semi-duplicate function logic, so it's a lot more code and execution paths become hard to conceptualise
-10%: it mirrors business logic it shouldn't, and subtly changes it. Which needs a huge amount of intervention to fix.
- 5% it goes completely off the rails. Eg decided that business/application logic should be in the database.
Prompt and check this honestly works very well (still waiting on tests with more than a handful of paying customers however!) but a FULLY automated agent pipeline? I'm not convinced. That last 15% of cases will make a degenerate codebase real fast.
Also without a CC subscription, paying per token, it would be horrifically expensive for my approach.
- notatoad
This seems like a useful write up of the concepts involved here, but I just hate reading Claude’s writing style so much.
This is content marketing, at least give it a review pass before you hit publish… if your marketing is unpolished AI slop, I have to assume your product is too.
- joshheitzman
We've had software factories for decades. They're usually called compilers, linkers, toolchains, etc.