Why coding agents aren't architects

We have been building several applications with coding agents over the last few months where we let the agent write the code without really looking at it. So we actually tried to live “Give up control over the code”. And it was very appealing at first, because the speed was incredible. But what shows after a while is an inconvenient truth.

Classes accumulate. Whatever needs adding lands where something similar already sits, because that is the placement least likely to break anything — and the larger a class grows, the more likely the next thing lands there too. So you end up with a couple of 2000 - 3000 line classes and a bunch of minor ones. Extensions get bolted on from the outside rather than worked in, because integrating requires understanding the internals and bolting on does not. Special cases arrive as conditions, and the same distinction gets re-tested at three levels of a call chain, passed down through flag parameters, until the decision exists in no single place. Inheritance is never used, instead a dozen similar solutions to a requirement are implemented. In short - the code is less structured than we had wished and with every new feature it gets worse.

None of it is careless. Each step is the minimal, correct, low-diff change that leaves the tests green. That is precisely the problem: everything the loop can measure is satisfied, and the one thing it cannot measure is what erodes.

Most humans writing the same code would have stopped. Not out of discipline, but because by the third or fourth interacting condition the truth table no longer fits in their heads. That discomfort is the trigger for restructuring, and structure in code is largely what that discomfort produces — layers exist so that a reader is permitted to be ignorant of what lies below. An agent has a limit too, but it sits higher and, more importantly, it does not announce itself. Overload does not produce doubt; it produces confident nonsense. What is missing is not the capacity to abstract, at least not totally. It is the pain that makes a human do it.

The consequence is measurable, and it is where the practical damage sits. Past a certain size, the time and tokens needed per change stop tracking the size of the change. They start tracking the size of the system. Modest requests turn expensive, and eventually they get refused on grounds of effort.

Back to the two questions

In the first post I raised two questions:

• How can I give up control over the source code?
• How can I stay in control of what the software actually does?

I treated the first as the economic one and the second as the hard one. What the decay shows is closer to the reverse, and it is what tempted me into reading them as a single question - that control over what the software does is identical to control over the code.

Losing the code does not announce itself as a structural complaint. It arrives disguised as a functional one. Nobody can say where a thing lives. The vocabulary in the code has drifted away from the vocabulary of the domain, so there is nothing left to point at - which means you can’t prompt the agent properly any more. The cost of each change has climbed past what the request is worth. At that point control over what the software does is gone — not because any requirement was implemented wrongly, but because you can no longer say where it should be different. Every symptom shows up on the functional side while the cause sits entirely on the structural one. When the first failure is only ever visible as the second, it is tempting to conclude they were never two problems. Tempting, and wrong.

Wrong, because they fail independently: a system can satisfy every requirement while being structurally lost underneath, and a system can be immaculately structured while implementing the wrong thing. Two failure modes, two kinds of evidence — one settled against behaviour, the other visible only in what the next change costs. Merge them and you end up with one acceptance step where you need two.

Where they meet is not in the question but in the answer. Reading did not stop working; it stopped keeping pace, and slowing down enough to read every line spends exactly the advantage the agent was there to provide. So neither question is answered by reading or writing code any more, and there is one grip left for both: constraints stated up front, structural as well as functional, plus mechanical verification that they hold. Same instrument, two distinct claims made with it.

Where the magic happens

The picture shows what AIND targets. Two registers, stacked. And two terms for the core concepts above and below the seam: Components on the intensional side, Modules on the extensional side.

The underlying statement here is: control must not disappear when using a coding agent to produce the code. It must transform from direct — write it yourself, or read and correct all of it — to indirect: state what must hold, and check it by machine. What changes is the level of detail at which you have to look, and the means by which you look.

If we accept this premise, it decides how much of this is achievable, because what we need then is methodology, language and tooling. They must enable us to write the specification in a way that is not only precise enough to correlate components (specification) clearly with modules (code), but also efficient so that it does not degenerate into another way of writing the code. Building these tools and the verification methods is the actual work.

Above the middle, everything is intensional. Architecture and components describe what must hold, what may vary, what is decided. Statements, not things. Below the middle, everything is extensional. Modules and code are the concrete extent — the set of things that actually exist. Refining architecture into components stays inside the intensional register; realizing modules as code stays inside the extensional one. Only one step in the whole stack crosses between them, and that is where the mapping sits.

That crossing is not symmetric - architecture theory has a name for this asymmetry, and it’s old news there: an intension determines an extension, not vice versa. The projection from the extension towards the intension is not unique, the same code may come from different intensions. Which means the correlation between the two cannot be reconstructed. It has to be written down before or while code is created — a first-class artifact, not a relationship a reader is expected to infer.

The vocabulary runs the full height because it is the medium in which the mapping is expressible at all. If components and modules do not share terms, there is nothing to map. This is also the mechanism by which control is actually lost in practice: when an agent coins its own names, the domain language and the code language become disjoint, and there is no longer any place to point at. So the vocabulary is a controlled artifact. New domain terms enter by admission, not by invention. Everyone who has used a coding agent with a powerful language model knows that this is a lot easier said than done. Assigning new meaning to well-known terms and making it sound perfectly reasonable while it is in fact gibberish once you take a closer look at it is one of the things where a frontier model excels.

The verification column is the load-bearing part. Above and below the seam, consistency is checkable, though not fully mechanical in every place. Above, you can identify contradictions between statements, rules that never fire, two terms that mean the same thing. Here an LLM combined with deterministic tools can be very helpful as long as you don’t treat it as the only source of truth. Below the seam, structure is checkable: dependency directions, placement, size budgets, the names in the code against the vocabulary. The latter is almost completely mechanical, and neither of both requires anyone to read the code.

And that is why the collaboration zone sits exactly where it sits. Not as a compromise between two extremes, but because the register change is the one place no check reaches. Bringing a component description and a module into coincidence — does this actually realize that? — is a judgement, and judgement is the thing models are least reliable at, therefore it requires human work and intelligence.

What is not settled

As nice as all this sounds, it only points at the problem, it does not yet solve it.

The resolution of the mapping. Component-to-module is coarse: it tells you which module realizes which component, and nothing below that. A finer mapping — responsibility to type, claim to member — would make change locality much sharper, but only if it is genuinely the same relation at a finer resolution rather than a second artifact with its own drift. That question has a factual answer and we have not measured it yet.

The economics. The whole argument rests on the claim that maintaining constraints grows more slowly than maintaining code. If that is false, AIND has moved the work rather than reduced it. Everything measured so far is quality — reproducibility, confirmation rates. The slope is still missing, and it is the one number the approach should be judged on.

Which brings this back to where the first post started. Agents did not make good software easier to build. They show more clearly what the hard part is, because the easier part can be done very fast by a machine now.