Context engineering is a budget, not a craft

Context engineering is deciding what an agent's context window holds on each call and, over a long run, what it gives up. The window is a budget. Tool results, history and retrieved files spend it. Quality falls as it fills, well before the limit. Every eviction has a price, and it rarely shows up where you look.

The explainers we read stop at the definition: the art of putting the right information in the window. That is true, and it does not help at turn fifty, when the window holds about 140,000 tokens and the agent may already be repeating itself. What helps is the accounting. What fills the window, what degrades as it fills, and what each eviction (clearing tool results, compaction, sub-agents, memory files) costs in tokens, cache and correctness.

The agent loop piece argued that the window is the agent's state machine. Accept that, and the next question is who keeps the books.

What fills the window on a long run

Two things sit in the window before the agent does any work: the system prompt and the tool schemas. They are fixed, and they are paid for on every call. The trajectory is everything else, and the trajectory is mostly tool results.

The team behind Manus published the shape of its own traffic in July 2025. Yichao "Peak" Ji's Context Engineering for AI Agents: Lessons from Building Manus puts a typical task at around 50 tool calls. Input tokens outnumber output tokens about 100 to 1. The agent reads far more than it writes, and each turn re-sends everything that came before, plus whatever the last tool returned.

Suppose a coding agent with an 8,000-token prefix runs 50 turns. Each turn it writes a 200-token tool call and gets back a 2,500-token result: a file read, a test log, a page of search hits. On its final call the window holds 140,300 tokens, and the fixed prefix is under 6% of it. Across the run the agent has sent 3,707,500 input tokens, because the prefix was paid for 50 times, the first file read 49 times, and so on down. This is the append-only baseline. It is what an agent does when nobody has made an eviction decision.

Spending tokens is also how an agent gets results. In Anthropic's write-up of its multi-agent research system, token usage alone explained 80% of the variance in performance on the BrowseComp evaluation. So the budget question is never "fewer tokens." It is what the tokens buy. A 2,500-token directory listing from turn six that is still in the window at turn forty buys nothing, and by then it has been sent 34 times. The dollar side of the ledger is its own piece: where agent tokens go.

Context rot starts long before the limit

The advertised window size is a ceiling. The usable budget is lower, and nobody prints it on the spec sheet.

Anthropic's Effective context engineering for AI agents (September 2025) gives the mechanism in one line of arithmetic. Every token attends to every other token, so n tokens make n² pairwise relationships, and attention spreads thinner as the window grows. The post calls this an attention budget, and it describes the decline as a gradient rather than a cliff.

Chroma measured the gradient. Its Context Rot technical report (Kelly Hong, Anton Troynikov and Jeff Huber, July 2025) tested 18 models on simple, controlled tasks. On 306 LongMemEval prompts, every model scored higher with a focused version of about 300 tokens than with the full version of about 113,000. Both held the answer. One distractor, a passage close to the answer but wrong, lowered accuracy; four lowered it further.

The families failed in different directions. Claude Sonnet 4 and Opus 4 often abstained when unsure, while GPT models had the highest rates of confident wrong answers when distractors were present. And across all 18 models, a needle was easier to find in a haystack of shuffled sentences than in a coherent essay. The report's conclusion is that whether the information is in the context matters less than how it is presented.

Position matters as well. Lost in the Middle (Liu et al., 2023) found that models often do best when the relevant information sits at the start or the end of a long input. On a long run, the middle is where most of the trajectory sits: the plan the agent made at turn three, the constraint it found at turn twelve.

Drew Breunig's How Long Contexts Fail gives the failures agent-shaped names, with the evidence he collected for each:

• Poisoning. An early hallucination enters the window and gets cited from then on.
• Distraction. The history grows until the model leans on it over its training. The Gemini 2.5 report's Pokémon agent drifted toward repeating past actions once its context grew well past 100,000 tokens; the report calls this anecdotal. A Databricks study saw Llama 3.1 405b's correctness start falling around 32,000 tokens.
• Confusion. Irrelevant material degrades the answer, unused tools included. A quantized Llama 3.1 8b failed a query with all 46 tools available and passed it with 19.
• Clash. New turns contradict old ones. In a Microsoft and Salesforce study that split prompts across several turns, performance fell 39% on average, and o3 kept only about 64% of its single-turn performance once the prompt was split.

Those thresholds come from other people's tasks, and they differ by model. Your agent has a threshold too, and the only place to find it is your own traces. Until you have, the advertised limit is the wrong number to plan against.

Sponsored:

Clearing old tool results: cheap in tokens, paid in cache

The lightest eviction is dropping raw tool results the agent has already used. Anthropic's context engineering post calls tool-result clearing one of the safest, lightest-touch forms of compaction: once a result is deep in history, the agent rarely needs the raw output again. The Claude API ships it as context editing. The September 2025 announcement reported the effect on a 100-turn web search evaluation. Agents finished workflows that would otherwise have run out of context, and token consumption fell by 84%.

The bill arrives through the prompt cache. Providers cache a prompt's prefix, and a change at any position invalidates everything after it. Ji argues the cache hit rate is the most important metric for a production agent. Its numbers show why. At the prices it quoted for Claude Sonnet, cached input cost $0.30 per million tokens and uncached input $3. Clearing an old tool result edits the middle of the prompt, and everything after the edit goes back to full price.

Here is the running example priced three ways. Append-only clears nothing. A sliding clear keeps the 5 most recent tool results in full and stubs the rest, on every turn. A batch clear waits until the window passes 60,000 tokens, then stubs all but the last 5. The script counts the cache as the longest prefix unchanged since the previous call.

# One 50-turn agent run, priced under three eviction policies.
PREFIX = 8_000        # system prompt + tool schemas, tokens
CALL = 200            # tool call the model writes each turn
RESULT = 2_500        # tool result appended each turn
STUB = 50             # placeholder left by a cleared result
TURNS = 50
KEEP = 5              # most recent results kept in full
TRIGGER = 60_000      # the batch policy clears past this size
CACHED, UNCACHED = 0.30, 3.00  # USD per million input tokens
 
 
def segments(kept):
    """The window as a list of segment sizes, in prompt order."""
    out = [PREFIX]
    for full in kept:
        out += [CALL, RESULT if full else STUB]
    return out
 
 
def keep_recent(kept):
    return [i >= len(kept) - KEEP for i in range(len(kept))]
 
 
def price(policy):
    kept, previous = [], []
    sent = peak = 0
    cost = 0.0
    for _ in range(TURNS):
        if policy == "sliding" or (
            policy == "batch" and sum(segments(kept)) > TRIGGER
        ):
            kept = keep_recent(kept)
        window = segments(kept)
        hit = 0  # cache covers the unchanged prefix since the last call
        for before, now in zip(previous, window):
            if before != now:
                break
            hit += now
        size = sum(window)
        sent += size
        peak = max(peak, size)
        cost += (hit * CACHED + (size - hit) * UNCACHED) / 1e6
        previous = window
        kept.append(True)  # this turn's tool result lands
    return peak, sent, cost
 
 
for policy in ("append-only", "sliding", "batch"):
    peak, sent, cost = price(policy)
    print(f"{policy:12} peak {peak:>7,}  sent {sent:>9,}  ${cost:.2f}")

On Python 3.12.3 it prints:

append-only  peak 140,300  sent 3,707,500  $1.49
sliding      peak  32,500  sent 1,282,000  $2.05
batch        peak  59,300  sent 1,916,550  $1.07

The sliding clear sends 65% fewer input tokens than append-only and costs 38% more, because it breaks the cache on every turn. The batch clear sends 48% fewer and costs 28% less: it breaks the cache now and then, and the rebuilt prefix pays for itself. The model leaves out cache-write surcharges, which make the sliding policy look worse, and cache expiry, which hurts the cache-friendly policies most.

Anthropic's context editing documentation carries the same trade-off as a parameter. Its clear_at_least setting skips a clearing pass that would free too little. In the docs' words, it "helps determine if context clearing is worth breaking your prompt cache." The defaults trigger at 100,000 input tokens and keep the 3 most recent tool uses.

Clearing also costs correctness. A cleared result becomes a placeholder saying it was removed, so an agent that needs it again re-runs the tool. That is fine for a file read and wrong for anything slow, paid, or not idempotent. Manus adds a rule worth copying into any clearing policy: leave failed actions and stack traces in the window. A model that can see its mistakes is less likely to repeat them. A policy that treats an error trace as one more old tool result throws away the evidence.

Compaction decides what the agent may forget

When clearing is not enough, the next step is compaction: summarize the history and start a fresh window from the summary. It is the cheapest strategy to describe and the hardest to get right.

Anthropic's context engineering post describes Claude Code's version. The model summarizes the message history, keeping architectural decisions, unresolved bugs and implementation details and dropping redundant tool output. The agent then carries on from the summary plus the five files it accessed most recently. The API documentation describes the SDK's version, now deprecated in favor of server-side compaction, more bluntly. Past a threshold, 100,000 tokens by default, the SDK injects a summary prompt as a user turn and replaces the entire message history with the answer.

The bill is fidelity. The summarizer is one model call deciding, in one pass, what the rest of the task needs. Anthropic's advice is to tune the compaction prompt for recall first and precision second, because an over-eager summary loses details that matter later. Breunig's taxonomy names the worse case. A summary that carries a hallucination forward is poisoning with a stamp of authority: the agent now reads its own error as settled history. And compaction gives up the whole cache, since the new window is a new prefix.

Two practices follow. Treat the compaction prompt as code: version it, review it, and run your eval suite when it changes (the evals piece covers how). Then log the full window before every compaction. In a postmortem, "what did the summary drop?" has no answer without it.

Sub-agents and memory files move the budget elsewhere

The last two strategies do not cut what is already in the window. They keep work out of it.

A sub-agent gets a clean window, does one focused job, and returns a summary. Anthropic's post puts the shape at tens of thousands of tokens spent inside the sub-agent and a condensed result of 1,000–2,000 tokens handed back to the lead. Breunig calls this context quarantine. The lead's window stays small because the tokens are spent somewhere else.

The bill is the multiplier and the seam. In Anthropic's data, agents typically used about 4 times the tokens of a chat, and multi-agent systems about 15 times. The same post says multi-agent designs fit poorly where agents must share context or depend on each other's work, and it notes that most coding tasks involve fewer parallelizable pieces than research. A sub-agent's summary is a compaction by another name, written by a model that never saw what the lead needed. Among the post's tips: for some kinds of output, have sub-agents write to external storage and pass back a lightweight reference. The post's name for what that avoids is a game of telephone through the lead. How the seams fail is a taxonomy of its own.

Memory files are the cheapest strategy on paper. The agent writes notes outside the window and reads them back later: Claude Code's to-do lists, a NOTES.md, Manus's todo.md. Anthropic's example is its Pokémon-playing agent, which kept tallies across thousands of game steps: "for the last 1,234 steps I've been training my Pokémon in Route 1 …" After each context reset it read its notes and picked up where it was. Manus uses the file for recitation as well. The agent rewrites its to-do list as it works, which moves the plan to the end of the window, away from the middle where it gets lost. Manus also treats the file system as restorable eviction: drop a web page's content but keep its URL, drop a document but keep its path, and the agent can fetch either again.

Memory files cost discipline. A note helps only if the agent writes it at the right moment and reads it at the right moment, and both are model decisions. A stale note is a wrong fact that survives every reset. The evidence that the combination pays is real and narrow. In Anthropic's announcement, context editing alone improved performance on its internal agentic search evaluation by 29%, and context editing plus the memory tool by 39%. That was Anthropic's task, measured by Anthropic.

The context engineering ledger

Side by side, each strategy saves window at a different price.

Every entry in the failure column looks, in a trace, like an agent forgetting something. That is what the policy told it to do.

The sources and the arithmetic support an order. Start with what never needed to enter the window: smaller tool results, fewer tools, pointers instead of content. Breunig's How to Fix Your Context cites a study where retrieving fewer than 30 tools per prompt improved tool selection accuracy up to threefold. Manus's advice belongs here too: mask tools rather than remove them mid-run. Tool definitions sit near the front of the prompt, and removing one breaks the cache for everything after. Then clear tool results in batches big enough to pay for the cache miss. Compact only when clearing is not enough, and only with a prompt you have tested. Use sub-agents when the work splits into pieces that do not need each other's decisions. Memory files can sit alongside any of these, for the facts that must survive a reset.

Monday

Pull the trace of your longest agent run from last week and chart the window size on each call, split into prefix, tool results, and everything else. Mark the turn where the agent first did something you would not have done. If that turn falls after tool results became most of the window, you know where clearing starts.

Then price your eviction policy the way the script does: count the cached prefix on each call, not only the tokens sent. If you clear on every turn, compare the bill against batching the clears behind a minimum. Last, check whether your clearing drops error traces. If it does, exempt them.