note · 2026-10-04

Three engineers, no one reads the code, and at least 1,000 dollars a day in tokens: the software factory that runs

One security company's own account of a software factory where code is neither written nor reviewed by humans. What replaced the reviewer was a set of scenarios the agents never see.

For founders and CTOs of SaaS companies deciding how much of their pipeline can run without a person reading the codeyaab1 min read
Three engineers, no one reads the code, and at least 1,000 dollars a day in tokens: the software factory that runs

A software factory, a pipeline that takes specifications in and produces code out, only works if you can prove the code works without reading it. One company's own account, not a study, describes how.

Simon Willison, creator of Datasette, described on February 7, 2026 the factory of a three person team at StrongDM, a security software company. The team was founded on July 14, 2025 and published its own account a day before Willison's. Its rules: code must not be written by humans, code must not be reviewed by humans, and a team spending under 1,000 dollars a day in tokens per engineer has room to improve. The first problem was agents writing tests to pass. Their answer: scenarios, end to end user stories stored outside the codebase like a holdout set, judged by a model on a satisfaction rate, and run by the thousands per hour against behavioral clones of the six services the software depends on. Willison's question: how do you prove software works when implementation and tests are both written by agents?

Dex Horthy, founder and CEO of HumanLayer, on X, joking that he said CI/CD instead of software factory and got kicked out of San Francisco.

In the SaaS lines we run, nobody reads every line either. Somebody owns the scenarios.

The factory is the validation. The agents are the cheap part.

Sources