The software industry's next survival test

Every few years the software industry declares a new existential threat. No-code will kill developers. The cloud will kill on-prem. Now coding agents will replace human developers.

At first glance it seems obvious that coding agents have the potential to widely replace humans when it comes to writing code. At least this is the conclusion our team, myself included, draws from more than 6 months of intense daily work with such tools. The capabilities and the pace at which they improve are stunning. What is less obvious is what remains as human work. Let us look at two key questions that have to be answered by the software industry facing the increasing capabilities of AI coding agents.

1. How can I give up control over the source code?
2. How can I stay in control of what the software actually does?

A consistent answer to both questions simultaneously is what will define how we will develop software with coding agents - not either question in isolation, but the two together, because they pull in different directions and any real methodology has to hold both at once.

Whether such an answer will actually emerge - whether agent-driven development becomes the norm at all - is a separate, secondary question. Either it exists, and coding agents can be used at scale in a way that’s economically and operationally responsible, or it doesn’t, and relying on them long-term becomes irresponsible on both counts. My side bet: given how much is economically at stake, betting against it would be the more surprising position.

Now let’s take a closer look at the two questions. The first one basically has an economic motivation: To leverage the cost advantage of using a coding agent instead of writing the code manually, you can not afford to try and read every line the agent created. It will just take too much time and you will lose much of the speed advantage the agent can give you.

The second question, however, is not primarily economic in nature. It is about reliability and predictability, about the user’s intention for the software, which becomes the developer’s intention (hopefully). And it has many facets:

• pure functionality (does the software support a certain workflow or process at user level)
• performance (are the response times within an acceptable range)
• maintainability (is the structure comprehensible, do changes stay local)
• efficiency (can the task at hand be solved with a reasonable amount of resources)

We could go on for quite a while with this list.

In the past, the second question was answered by reading the code and trying to understand what it does. As pointed out, this is not going to work any more. So the second question comes down to: How do I tell the coding agent to produce software that fulfills my requirements and how do I verify that it did? And this turns out not to be trivial or straightforward for many reasons, but that insight is not exactly new. It has been discussed in research for more than 40 years - it goes back to Dijkstra’s work on layered system structure in the late 1960s and Parnas’ work on decomposing systems into modules in the 1970s, back when developers first started building systems too large for any one person to fully hold in their head. It was a multi-faceted problem right from the start, involving linguistic aspects as well as engineering and design aspects.

In this blog I will try and dive deeper into some of these issues as they arise during our development of AIND - this is what our team called the methodic approach to AI based software development. AIND stands for AI-native development as opposed to vibe coding. AIND targets giving up direct control over the code entirely, reducing the human role to specification and verification only.