Postmortems for something nobody typed

An incident review opens with a timeline, and the timeline opens with an action. For most of the history of the discipline, the action had a person attached to it: someone typed the command, merged the diff, clicked the button. The postmortem's craft was in getting past that person to the system that made their action reasonable.

Now some of the actions have nobody attached. A diff was produced by a process that read the ticket, browsed the repository, and decided. A person approved it, or a policy did. The review room's oldest instinct, find the hands, has nothing to hold.

When the actor is a loop, the review is for something else: what the team decided to trust, and what it learns to control once it stops looking for hands.

Reconstruct the trajectory first

A human's account of what they did is unreliable in the well-documented ways; it is still an account, and you can ask follow-up questions. An agent leaves no account unless you built one. If the trace was not captured, the trajectory is gone, and the postmortem is reduced to inspecting the artifact and guessing.

So the first finding in most agent postmortems is about tracing, and it tends to be the same finding: the team logged the final output and the tool calls but not the context at each call. They know what the agent did and cannot know what it saw. An agent that removed a config block after reading a file that said the block was unused made a defensible decision on bad information. An agent that removed it without reading anything made a different kind of error. From the outside, the diffs are identical.

Replay is the second requirement. Given the frozen context at a step, can you re-run the step and watch it decide? Nondeterminism means the replay may not reproduce the exact action, and that is itself information. A decision that recurs on replay is a systematic behavior; a decision that does not is a low-probability path you may or may not choose to defend against. Neither is available without the capture.

Charity Majors has spent years arguing that you cannot debug what you cannot observe, about ordinary distributed systems. Agents make the argument sharper, because the interesting state is the window, and the window is discarded at the end of the call unless something saves it.

Authorship versus judgment

The instinct to ask "who wrote it" comes from a world where writing and deciding were the same act. The engineer who typed the line also judged that the line was right. Accountability could attach to the keystrokes because the judgment was inside them.

An agent separates the two. The writing is done by a process that has no stake and cannot be held to account in any sense that changes its behavior next week. The judgment happened elsewhere, and earlier: when someone decided that this class of task could be delegated to this agent with these tools and this level of review. That decision was made by a person, was reasonable or not on the information available, and is the thing the postmortem is about.

Accountability follows the decision to trust, not the keystrokes. The team that configured the agent to auto-merge diffs under a size threshold made a judgment. The reviewer who approved the diff made a judgment. The engineer who wrote the tool description that the agent misread made a judgment. None of them typed the line. All of them are in the timeline.

This is not a way of finding someone to blame after all. John Allspaw's work on blameless reviews, and Sidney Dekker's "new view" of human error before it, established the point of locating a human judgment: understand why it made sense at the time, then change the conditions rather than the human. The same holds here, with one addition: some of the conditions are now prompts, tool descriptions, and policies, and those can be changed with a pull request.

The three questions

In place of "who wrote it," the review asks three things, in order.

What did we decide to trust, and on what basis? Which tasks, which tools, which level of autonomy, which review gate. Was that decision written down? Did the incident fall inside its stated scope, or did the agent do something the trust decision never contemplated? The second case is a scope failure, and the fix is a boundary, not a better prompt.

What did the agent have in its window when it acted, and what did it lack? This is the trajectory question, and it is only answerable with the trace. Most agent errors turn out to be errors of missing information: a truncated tool result, a summarized-away constraint, a file it never read because the search tool ranked it 51st. The fix is to the information path, and it generalizes to every future task that walks the same path.

What bounded the blast radius, and was it the boundary we intended? Sometimes the answer is "nothing; it stopped because it happened to stop." That is the finding that reorganizes a team's priorities, because it means the only thing between the incident and a worse one was the agent's own judgment, which is exactly the component under review.

None of these questions has a name in the answer. All of them produce a change.

Sponsored:

What the approver actually verified

Most agent workflows have a human approval step, and the postmortem needs to be precise about what that step verified, as opposed to what it was assumed to verify.

The observed behavior across teams is consistent. A reviewer receives a diff that an agent produced, along with a summary the agent wrote. The reviewer reads the summary, skims the diff, sees that CI passed, and approves. Asked later, the reviewer says they verified the change. Asked what they checked, the honest answer is: that the summary was plausible and the diff was not obviously wrong.

That is not negligence. It is what approval looks like when the volume is high and the artifact arrives with its own persuasive description. The agent's summary is written by the same process that made the error, and it is confident, because the process does not know it erred. The reviewer is reading the defendant's account of events and finding it convincing.

The postmortem's job is to state, without judgment, the gap between the verification the process assumed and the verification that occurred. Then it can ask whether the gap should be closed by asking more of the reviewer (a losing strategy at volume) or by moving the verification into something mechanical: an eval, a policy check, a sandbox that makes the wrong action impossible rather than merely disapproved.

Blast radius is the control

If judgment is expensive and approval is porous, what is left is the boundary: the set of things the agent cannot do regardless of what it decides.

Sandboxing is the discipline, and it has three walls. Filesystem: the agent writes inside a directory it owns, and nothing else, and the boundary is enforced by the operating system or the container, not by a tool description that asks nicely. Network egress: the agent reaches the hosts on an allowlist and no others, so that whatever it reads cannot instruct it to send data somewhere. Simon Willison's "lethal trifecta" (access to private data, exposure to untrusted content, a channel to exfiltrate) is the crispest statement of why egress is the wall that matters most. Remove the third leg and most of the injection stories lose their ending. Credentials: the agent holds tokens scoped to the task and expiring with it, never a person's session, never a deploy key that outlives the run.

Against that list, the permission prompt is the wrong safety layer. A prompt asks a human to evaluate a single action with less context than the agent had and less time than the agent took. Repeated dozens of times an hour, it trains the human to approve. And it protects nothing the agent can route around: an agent denied one path to a goal tries another, and the second path may not have a prompt on it. The prompt is a UX affordance for the human's comfort. The boundary is a control. Teams that confuse the two discover the difference in the postmortem.

The permission prompt is for the human's comfort. The boundary is the control. Allow the autonomy inside the walls to be wide; that is what the walls are for.

An agent that can do anything to its own working copy, on its own branch, in a container with no route out except through a review, is safe to let run, and useful because it is running.

Prompts, tools, and policies are code

Every postmortem produces a change. In an agent system the change is often to a prompt, a tool description, a compaction policy, or an approval threshold. Those artifacts have to be versioned, reviewed, tested, and revertible, or the postmortem's action items evaporate into a text field that someone edits in place.

The practical requirements are the ones you already apply to code. The system prompt lives in the repository and changes through review. Tool descriptions are part of the tool's source, and a change to a description is a change to the API. Policies (what auto-merges, what requires approval, what step budget applies) are configuration under version control with a deploy history. The eval suite runs on every change to any of them, so that a prompt edit made in response to Tuesday's incident does not silently reintroduce March's.

Rollback is the requirement teams forget. When a prompt change goes wrong, the fix is to revert the prompt, which is only possible if the previous version exists somewhere other than someone's memory. The observed behavior in teams without this is a series of prompt edits, each addressing the last edit's side effect, converging on nothing.

An illustrative walkthrough

What follows is invented, to show the shape. Suppose a team runs a maintenance agent that handles a queue of small tickets, with a policy that diffs under 200 lines touching only non-production paths auto-merge after CI, and larger diffs go to a human.

A ticket asks the agent to remove feature flags that have been fully rolled out. The agent lists flags, calls a search tool to find references to each, finds no references to one of them, and removes its definition. The diff is small and touches only the flags module. CI passes. The policy auto-merges. Two hours later, a regional deployment that was still gated on that flag starts serving the new behavior to a cohort it was not meant to reach.

The review begins with the trace. The search tool, it turns out, returns at most 50 results, ranked by path, and the regional config that referenced the flag was the 53rd. The agent's window contained a result that said, in effect, no references found. On that information, removal was correct.

The three questions, then. What did the team trust? The policy trusted "small diff, non-production path, CI green" as a proxy for safe, and the flags module was classified as non-production because it was not a service. What did the agent have? A truncated result that presented as complete. What bounded the blast radius? The regional rollout's own gradual ramp, which was an accident of unrelated design and not a boundary anyone chose.

The changes follow without anyone being named. The search tool now reports when results were truncated, and by how much. The flags module is classified as production. The auto-merge policy gains an exclusion for anything that alters runtime configuration. An eval case is added: a flag with a reference beyond the search tool's page size must not be removed. The reviewer who would have caught it never saw the diff, and the postmortem notes that this was the policy's decision, not the reviewer's.

The invented part is the details. The shape recurs.

Where judgment sits, and what happens when it moves

Human judgment does not leave an agent workflow. It moves. It moves from the line to the task, from the diff to the policy, from "is this change right" to "is this class of change safe to delegate with these boundaries." The people who hold that judgment are fewer, and the consequences of their decisions are wider, because a policy applies to every run and a prompt is read by every task.

The observed effect on teams is a change in what seniority means. The engineer who used to be valued for writing the right code is now valued for specifying the boundary correctly, reading a trace quickly, and knowing which kinds of task the agent gets wrong. Junior engineers, who used to learn judgment by writing the code and having it reviewed, are now approving code they did not write, which is a slower and stranger way to learn. Teams that notice this early rotate people through the trace-reading and case-writing work deliberately, because it is where the understanding of the system now accumulates.

The ominous version of the finding is that a team can drift into holding no judgment at all: the policies were set once, the approvals became reflexive, and the only component deciding anything is the one that cannot be held to account. The postmortem is where that drift becomes visible, which is one more reason to hold one for every incident with nobody's hands on it.

Monday

Take the last incident that involved an agent, or the agent-produced change you most nearly regretted. Answer the three questions in writing: what was trusted and on what basis, what was in the window at the decisive step, and what bounded the damage. If any of the three is unanswerable, that is the first action item, and it is about tracing or boundaries, not about the agent.

Then find the permission prompt you rely on most and ask what would happen if the person clicked approve without reading. If the answer is "the same thing that happens now," you have found your safety layer, and it is not the prompt.