Dan Shapiro describes the top level of AI coding as "a black box that turns specs into software" (The Five Levels, January 2026). StrongDM's operating rule is blunter: "Code must not be reviewed by humans," as Simon Willison reported in February. If humans do not read the code, what checks keep each change within the system's intended architecture? In the factory accounts I read this year, I could not find a clear answer.
What the factory builders got right
The factory builders have moved verification away from reading diffs and towards checking behaviour. That is useful work.
StrongDM's version, as Willison's post quotes it, is the clearest. Its scenarios are end-to-end user stories, "often stored outside the codebase", like a holdout set in model training. That keeps the agents writing code from writing their own tests.
StrongDM measures success with what it calls satisfaction: the share of observed scenario trajectories likely to satisfy the user. Its software talks to other services, so the team built a Digital Twin Universe: "behavioral clones of the third-party services our software depends on". Okta, Jira, Slack and Google apps are among them. The scenarios can run thousands of times an hour against the clones.
That is a serious answer to the question "does this software do what its users need today?" I see no reason to question that approach.
Every check is about one change
A different question remains: will the system still be in a shape anyone can work in next year?
Two surveys of factory builders appeared this summer: Jina Yoon's for PostHog (2026-08-11) and Andrew Zigler and Ben Lloyd Pearson's for Dev Interrupted (2026-08-13). Both describe specs, requirements and sometimes design review for each change. Neither describes a standing view of the whole system checked against each run. The checks they discuss focus on the change and its spec.
Both surveys quote Dex Horthy on one risk: coding agents are rewarded in training for reaching a correct answer. As he puts it in the PostHog piece:
If the tests pass, we win, but there is no penalty for eroding codebase maintainability.
Dev Interrupted makes a related point: tests can pass on code "that's quietly harder to change than what it replaced."
Addy Osmani also describes how context-window limits can hide constraints elsewhere in a large codebase from an agent (his essay on the factory model).
Illustrative: the checks compare each change with its spec; the articles do not describe comparing it with the whole system.
Together, these examples point to a gap. A change can pass its own tests while its dependencies move outside the architecture the team intended. A standing view of the system, checked alongside each change, would make that drift visible.
When the system changes faster than people can follow
On a small team, the people deciding what to build may also be the people writing it. Shapiro describes a handful of level-five teams he knows as having fewer than five people; that is a small sample, not a rule about factories generally.
Agents increase the pace of implementation, which makes it harder for anyone to follow the whole system as it changes. LinkedIn's CPO says the company replaced its Associate Product Manager program with an Associate Product Builder program that teaches coding, design and product together. That is one example of roles converging, not evidence of how often factory changes outpace a team's understanding. A shared, current view of the system gives people a way to keep that understanding in step with the work.
What a floor plan is, and what it costs
I think a factory needs an architecture model a check can read before the agents start, and each run should be checked against it as well as its own spec. Osmani makes the case for planning ahead: "Poor architectural decisions made upfront do not affect one implementation. They propagate across the entire fleet."
In practice, the model needs the system's parts, its allowed dependencies, the contracts other parts call, and the decisions behind those boundaries, each with an owner who can change it.
Illustrative: six months of green runs can still leave two dependencies outside the agreed design.
This takes time. The team has to define the rules and settle disagreements about them, then keep the system model current. A structural check also has a limit: it can show when an agent connects parts the architecture keeps separate, but not whether the logic inside either part is correct. That still needs scenarios, tests or a person.
Choose the boundaries before the agents run
Inside enterprises, I expect the step just short of a dark factory, a person writing the spec and judging what comes back, to arrive quickly for internal apps. I don't know whether fully dark factories will hold up for systems that have to live for years. Horthy says he would feel better if models scored "80%+ on a (well-held-out) benchmark like SlopCodeBench," as Dev Interrupted quotes him. I would watch for that result. Until then, each team has to decide how much to automate. I would start by writing down three or four boundaries an agent must not cross, then deciding how to test them.
ProvenMap works on that system-wide view. Boards combine real sources, and the code plugin can derive a board from a repository. On an app board, boundary rules record allowed dependencies; on a successful sync, the code plugin marks each rule Holds or Violated and shows evidence of any crossing. Decisions carry an owner, and Build hands them to the coding agent as context.
Murali Krishna Ganta is the founder of ProvenMap, a software architecture intelligence platform. He previously built low-code platforms and worked on industrial digitalisation projects.
Sequoia Capital identifies a $10 trillion services automation opportunity, but calls this the "steam engine moment"—the foundational AI technology exists, but we're still decades from systematic deployment. Early agent-first implementations show 40-85% efficiency gains, yet success requires architectural expertise concentrated among technical specialists. The companies that democratize this knowledge—making agent system design as accessible as ChatGPT—will become the Westinghouse of the cognitive revolution.
The software architecture community is undergoing a quiet revolution. Industry leaders report a fundamental shift from traditional "ivory tower" architects who control decisions from above, toward collaborative enablers who empower teams through coaching and guidance. This evolution isn't diminishing the architect's role—it's expanding their impact.
An engineering retrospective from 2025. We prototyped a type-safe agent-workflow builder, then cut it — because the harder question turned out to be the simpler one: what is actually in my system, and what will this change break?