The agent loop fits on a page; production does not

The demonstration is by now a genre. Thorsten Ball's version, How to Build an Agent, runs in a few hundred lines of Go: a chat loop, a handful of tools (read a file, list a directory, edit a file), and a model that decides when to call them. You watch it fix a bug and something in you recalibrates. There is no orchestration layer and no planner. There is a for loop.

That recalibration is correct, and it is also where the trouble starts. Teams read the loop, build the loop, ship the loop, and then spend the next two quarters discovering that none of the work was in the loop. It was in what the loop can see, what it is allowed to touch, and what happens when you run several of them at once.

The loop is the organism at rest. Production is the organism under load, and it behaves differently.

The organism at rest

Strip any agent framework to its skeleton and you find the same four parts. A model. A list of tools, each with a name, a description, and a schema. A context: the conversation so far, including every tool call and every result. And a loop: send the context to the model; if it returns a tool call, run the tool, append the result, go again; if it returns text, stop.

Ball's post shows this in the plainest terms. Anthropic's Building effective agents gives it a vocabulary and, more usefully, a warning. It separates workflows (code decides the control flow, the model fills in steps) from agents (the model decides the control flow) and then spends most of its length arguing that you should reach for the least of these that solves your problem. The ReAct paper (Yao et al., 2022) is the origin of the interleaved reason-act-observe pattern the loop encodes; it is worth reading to see how much was assumed about the environment being cheap to query and the observation being small enough to keep.

Notice what the skeleton does not contain. No memory beyond the context. No notion of a task other than the first user message. No budget. No way to tell the difference between "I am done" and "I have run out of ideas." Every one of those gaps gets filled in production, and each filling is where the complexity lives.

Four structures, and what each one eats

Teams do not stay on the bare loop. They graduate, usually under pressure, through a recognizable sequence. Here is the sequence with the bills attached.

Single loop with tools. One context, one model, N tools. The whole trajectory is one transcript. Debugging is reading. Cost scales with the length of the task, and it scales superlinearly: each turn re-sends the whole context, so a 50-turn task pays for its early turns 50 times over unless caching absorbs it. The ceiling is the window. When a task needs more history than fits, the loop either truncates and forgets or summarizes and distorts.

Workflow with LLM steps. Your code owns the control flow: parse the ticket, then classify, then draft, then verify. Each step is a bounded model call with a bounded context. This is the structure Anthropic's post keeps steering you back toward, and the reason is legible in the bills. Tokens are predictable because each step's input is designed rather than accumulated. Latency is the sum of the steps, which you can see. Debuggability is excellent: a failure has a step name. The cost is rigidity. The workflow handles the tasks you anticipated and degrades sharply on the ones you did not, and the model has no way to tell you it needed a step you did not build.

Planner and workers. One model call produces a plan; separate calls execute the pieces, each with its own fresh context; something collects the results. This buys parallelism and bounded contexts at the price of a new failure surface: the seam. Workers cannot see each other's decisions. A worker that resolves an ambiguity one way and a sibling that resolves it the other way both return "success." Cognition's team wrote up this failure under the plain title "Don't Build Multi-Agents," and their argument reduces to one observation: actions carry implicit decisions, and decisions made out of each other's sight conflict. Token cost multiplies by the number of workers, plus the plan, plus the merge. Latency is the slowest worker plus the plan plus the merge. Debugging now spans several transcripts and a coordinator, and the interesting bug is usually in the gaps between them.

Fleets. Many agents, each with a role, exchanging messages. The diagrams are handsome. The bills are the worst of the previous row multiplied by the number of edges. Anthropic, describing its own multi-agent research system, put the token multiplier at roughly fifteen times a plain chat interaction, by its own account and for its own workload; your number differs, but not in direction. Coordination is now a distributed-systems problem with a nondeterministic participant at every node. Debuggability drops to reconstructing a partial order of messages after the fact. Fleets earn their keep when the task genuinely decomposes into independent, parallel, wide searches, which is a narrower set of tasks than the diagrams suggest.

Read the table from the right. The failure column is the one that shows up in your incident channel; the other columns are what you paid to get it.

Sponsored:

Context is the state machine

Ask an engineer where an agent's state lives and they point at the framework's memory object. Ask the model and it points at the window. Only one of these is right.

The agent's state is exactly what is in the context at the moment of the call. Not the database, not the vector store, not the file on disk: those are places the agent can fetch state from, if it decides to and if the tool exists. What determines the next action is the tokens in the window, and the set of tokens in the window is a thing you are managing whether or not you have admitted it.

Three regions of that window deserve names.

What is there. The system prompt, the tool schemas, the task, every tool result so far. This is the part teams inspect. It is also the part that quietly bloats: a directory listing here, a 4,000-line file read there. A retrieved document that seemed necessary at turn six is still being paid for and still being attended to at turn forty, whether or not it matters anymore.

What got summarized away. When the window fills, something compacts it. The compaction is a model call, and a model call is a lossy function. What survives is what the summarizer considered salient, which is not the same as what the task considered salient. The agent that lost a constraint to compaction does not know it lost the constraint. It proceeds with confidence, because from inside the window nothing is missing. Anthropic's later writing on context engineering names the mitigations (compaction, structured note-taking, handing work to sub-agents with fresh windows), and each of them is a design decision about what the agent is permitted to forget.

What the agent can no longer see. This is the region nobody instruments. A worker's context is not the planner's. A tool result that was truncated to 10 KB had the answer 11 KB in. The observed behavior is an agent acting as though a fact were absent; the finding, on replay, is that the fact was absent from the window, though present in the system.

Once you accept the window as the state machine, several practices stop being optional. You log the full context of every call, not just the messages the user sees. You treat compaction prompts as code with tests. You budget the window like memory on an embedded device, because that is the shape of the constraint.

Tool design is API design

A tool is a function the model can call. The model reads the name, the description, and the schema, and infers the rest. Everything a tool's API says, and everything it leaves unsaid, becomes model behavior.

The observed pathologies are consistent across teams. A tool with an ambiguous name gets called for the wrong purpose. A tool whose description says "search" but whose result is a ranked list of ids gets called, then the ids get pasted into the next tool as if they were content. Two tools with overlapping scope produce a coin flip on every call. A tool that returns 80 KB of JSON teaches the model to stop calling it. A tool whose error message is error: 500 teaches the model to retry until the budget is gone.

The design rules that emerge look like the rules for a public API, because that is what this is, with a consumer that reads documentation literally and never opens the source. Name the tool for the intent, not the mechanism: find_owner_of_service, not query_graph. Return what the next step needs, in the form it needs it, and nothing else. Make errors sentences that say what to do instead. Make destructive tools distinct from read tools at the name level, so that the model's habit of pattern-matching on names works in your favor. Keep the tool count small enough that the schemas do not become the largest thing in the window, and audit that periodically, because tool lists only grow.

The second rule is less visible. Every tool result is a context write. A tool that returns generously is spending the agent's memory. The naturalist's observation here is that agents with wide tools and narrow windows behave erratically late in a task, and the finding is that the tools filled the window with things the task did not need.

How each structure fails

Each structure has a failure it is prone to. Naming them by shape helps you recognize which one is in your logs.

Loops. The single-loop agent calls the same tool with the same arguments, gets the same result, and calls it again. Sometimes it is a genuine oscillation between two fixes that each break the other. More often it is a tool result the model could not parse into progress, so it tried the last thing that produced a result. The cheap fix is a step budget. The real fix is a tool result that explains itself.

Drift. Long trajectories slide away from the original task. The mechanism is the state machine above: the task statement is one message among hundreds, and its relative weight in the window declines every turn. The agent optimizes for what is recent. You see this as an agent that finished something adjacent to what was asked, with a plausible rationale. Re-anchoring the task periodically (re-inserting it, or keeping it in a structured scratchpad the agent updates) is the standard countermeasure, and it is a context-management decision, not a prompt tweak.

Silent consensus. Planner-and-worker and fleet structures fail by agreement. Worker A makes an assumption; worker B, given the same ambiguous brief, makes the same assumption because it is the same model with the same priors; the merge sees two consistent outputs and passes them. Nothing disagreed, so nothing was flagged. This is the failure that arrives latest, because there is no error to catch; the output is wrong in a way that only the missing information could reveal. A reviewer model with a different brief, or a deliberate adversarial pass, is the countermeasure, and it costs another model call per merge.

Runaway cost. Every structure can do this, but fleets do it fastest. A retry policy interacting with a step budget interacting with a worker spawning workers is an exponential you did not intend to write. The observed behavior is a bill; the finding is usually that no single component had a budget, only the components' ancestors, and the ancestors were not watching.

Loops are loud. Consensus drift is silent. Order the failures by how late you find out; that order is the observability you need before you can afford the structure.

The simplest structure that works

The design stance: run the simplest structure that solves the task, and graduate only when you can name the symptom that forced you.

The symptoms are observable, and worth writing down as gates.

You leave the bare loop for a workflow when the loop is doing the same five things in the same order on most tasks and getting one of them wrong on a recurring fraction. The code can own that order. The model can keep the remainder.

You leave a workflow for planner-and-workers when tasks cannot fit one window and cannot be decomposed ahead of time, so the decomposition itself has to be a model decision. The pieces must also be genuinely independent: a worker does not need to know what another worker decided. If the pieces share decisions, you have a single loop with extra steps, and you should go back.

You go to a fleet when the task is a wide search over independent leads (research, exploration, enumeration) and the value of parallelism exceeds the token multiplier and the coordination debt. You do not go to a fleet because the diagram looks incomplete without it.

At every graduation, the tokens are the visible cost. The cost that matters is the loss of a single transcript you can read. The tracing, replay, and per-context logging that were nice-to-have on the bare loop become the precondition for understanding anything at all. A team that graduates before it can replay a run has traded a system it could debug for one it can only observe from the outside.

Monday

Take one agent you run and write down its four parts: model, tools, what is in the context at the first call, and the stop condition. Then write down which of the four structures it is, as opposed to which one the framework's README describes. Most teams find they are running a single loop wearing a planner's clothes, or a workflow that a framework has dressed up as autonomy.

Then pull a trace of a run that went wrong and answer one question: at the turn where it went wrong, what was in the window? If you cannot answer, that is the finding. The loop was never the hard part. The window was, and it still is.