There is a question that gets asked after every serious production incident, usually within the first hour: who approved this?

It is not asked to find someone to blame, or at least it should not be, but because the answer tells you where to look next: who understood the change, what they checked, and what they believed was true when they let it through. In a well-run engineering organisation that question always has an answer, and the answer is where the learning starts.

In AI-assisted delivery I keep finding that the question has quietly stopped having an answer at all.

I wrote in a previous post about the governance work required to make AI-assisted development trustworthy, and I touched on accountability as one of three failure modes I see regularly. It deserves more than a paragraph, because of the three it is the one that stays invisible longest. A missing specification announces itself within weeks, as drift and rework. Trusting the model too readily announces itself the first time a piece of code that reads well and passes its tests turns out to be doing the wrong thing. A missing accountability chain announces itself only after an incident, in front of the people asking the question, which is the worst moment to discover it.

How the chain breaks

Take a change that looks harmless, such as adding a retry to a payment call that has started timing out occasionally. An engineer describes the problem to a coding assistant across a long session, and what comes back is a retry with exponential backoff, three attempts, sensible logging and a test that passes. It reads well, it is the pattern anybody would have written by hand, and it goes into a pull request where a reviewer reads it the way they would read any small resilience fix and approves it.

Six weeks later a supplier reports being paid twice for the same invoice. The retry was correct in isolation, but the payment call was never idempotent, so a timeout that arrived after the payment had actually been taken produced a second one. Nobody in the chain was careless: the pattern was right, the test was right, and the flaw is in the interaction between the retry and a property of the endpoint that neither the model nor the engineer nor the reviewer checked.

Walk that chain backwards and it becomes surprisingly hard to say where accountability actually lives, because every link has a reason why it is not the one.

The model produced the code, but the model is not accountable for anything. It cannot be, in any meaningful sense: it holds no responsibility, learns nothing from your incident review, and will cheerfully produce the same class of defect tomorrow.

The engineer accepted the code, and would probably have written the same thing unaided, so the gap is not comprehension of the retry but knowledge of how the endpoint behaves. That gap is easy to miss at pace, because the volume and pace of AI-assisted work make deep comprehension of every line uneconomic, and because the whole premise of the tooling is that you can move faster than that.

The reviewer approved the pull request, having read it the way we read human-written code, which assumes an author who understood what they wrote and can be asked why. With AI-generated code that assumption no longer holds, and in the review practices I have seen it has not yet been written into how reviews are done.

The prompt, which is the nearest thing to a statement of intent, is sitting in a chat history that nobody kept, because a prompt is not documentation and was never written to be read again.

Each person in this chain behaved reasonably, and the chain still fails, because responsibility was distributed in a way that nobody designed and nobody agreed to.

Why accountability earns its keep

It is tempting to file this under governance formality, the sort of thing that matters to auditors and nobody else. I think that badly underestimates what accountability does for an engineering organisation.

Accountability is the mechanism by which organisations learn, because when an incident review can establish who made which decision with what information, you can fix the process that let the defect through. When it cannot, the review produces either a vague commitment to be more careful or a scapegoat, and the second actively teaches people to hide how the work really gets done.

There is also a harder-edged version of this argument, which comes from the years I spent working in and around regulated financial institutions, where “who approved this?” is not a retrospective question but a precondition for shipping anything. Those organisations solved accountability for human-written code decades ago, with review records, approval gates, and named owners. What strikes me in conversations now is how many of them are allowing AI-assisted changes to flow through those same controls without asking whether the controls still measure what they were designed to measure. A review record that says “approved” means something different when the approver was the only human who ever read the code, and read it at pace.

I doubt any regulator will accept “the model wrote it” as a category of explanation. A coding assistant is not itself a high-risk system under the EU AI Act, so the Act does not govern this directly, although the shape of its thinking is instructive, because it puts the obligations on the organisation deploying a system rather than on the system, and it requires human oversight by people with “the necessary competence, training and authority, as well as the necessary support”. That last phrase is the whole argument of this post in a regulatory clause. Accountability for AI output lands on the organisation whatever happens internally. The question is whether you have decided in advance where it lands, or whether the incident decides for you.

Two responses that fail

Two responses to this problem come up repeatedly, and I think neither of them works.

The first is the blanket declaration: “engineers are fully accountable for everything the AI produces”. As a sentence in a policy document it is hard to argue with, although it is usually issued without changing anything else, so the delivery pressure, the review capacity and the absence of a standard to review against all stay as they were. Accountability without the means to discharge it is liability transfer, and engineers know the difference, so the predictable result is that people stop declaring which code was AI-assisted and the visibility problem gets worse.

The second is the tooling reflex: the belief that a dashboard showing which lines were AI-generated solves the problem. Provenance data is genuinely useful, although knowing that a model wrote line 214 tells you nothing about who was supposed to verify it or against what standard, because accountability is a property of the process, and the telemetry only describes it.

What working accountability looks like

What follows is my own view of what works, and none of it is complicated.

Accountability follows acceptance, so the person who accepts AI output into the codebase owns it exactly as if they had written it, which is clear enough to act on and puts the model outside the accountability chain where it belongs, since the model cannot hold any part of it.

The approval record should say what was reviewed, against what standard, at what depth and by whom. That is what a specification gives you, something to review against, so that the word “approved” carries content, and a chat log does not do the same job however complete it is.

Review depth should be tiered honestly, because not every change needs the same scrutiny and pretending otherwise turns the scrutiny into theatre. Code that is public-facing, handles sensitive data or sits on a critical path wants review by an engineer with genuine expertise in that layer, whilst internal tooling can carry a lighter gate, and the tiers should be written down and agreed, since an unwritten tier system is individual judgement wearing a uniform.

The accountable person needs the means to do the job, which is the time to review the output properly, the specification to review it against, and the standing to say that something does not go out today. Delivery pressure that quietly removes any of those has removed the accountability with it, whatever the policy says.

This does cost something, and it would be dishonest to pretend otherwise, since a tiered review with real expertise on the high tier takes engineering time that would otherwise go into building. The part that costs least is the part most organisations skip, which is deciding who owns what and writing the standard down, because that is a decision rather than a process. What the whole of it buys is an incident review that can establish what happened, which is the difference between an organisation that improves after a failure and one that repeats it.

Familiar ground

All of this is familiar ground, because engineers shipping code they do not fully understand is not new, and we have been doing it for decades through third-party dependencies, copied snippets and frameworks treated as magic. The industry’s answer was never total comprehension but bounded trust, meaning maintainers, licences, security scanning, dependency review, and clear ownership of the decision to depend on something.

AI-assisted development is the same problem at a different volume, with output that is more plausible and less traceable than anything we have absorbed before, so although the scale is new the discipline it asks for is one the industry already knows how to apply.

When something AI-built breaks in production, “who approved this?” needs a real answer, and it is one of the eight questions in my governance self-assessment, where in my experience it is the one that produces the longest silence.