Context engineering for agent fleets

The context window is memory management. Everything the operating-systems literature says about working sets, eviction, and thrashing applies, and most agent frameworks are running without a policy.

An agent that works well for ten steps and poorly for eighty has not become less intelligent. Its working set has exceeded what it can attend to usefully, and the signal it needs is now buried under eighty steps of transcript.

Bigger windows moved the cliff; they did not remove it. Attention over a million tokens is not uniform, retrieval accuracy varies with position, and cost scales with what you actually put in there. The window is a budget, and budgets need policies.

Anatomy of a context#

For any long-running agent the window decomposes into five regions with very different economics.

| Region | Typical share | Volatility | Eviction policy | | --- | --- | --- | --- | | System / role | 3% | Never changes | Pinned | | Task statement | 2% | Never changes | Pinned, and repeated at the tail | | Tool definitions | 10–25% | Changes with phase | Swap by phase | | Working memory | 40–60% | Every step | Compact and evict | | Retrieved material | 15–30% | Every step | Evict on use |

Two of those rows are where the wins are.

Tool definitions are quietly enormous. Thirty tools with thorough JSON schemas is easily 8,000 tokens paid on every single call, and a model choosing among thirty options also chooses worse than one choosing among six. Load tools by phase: a research phase does not need the deployment tools in scope.

Working memory is where the degradation happens, and where a policy is worth the most.

Pin the task statement at both the head and the tail of the window. It is the cheapest single intervention in agent engineering, and it measurably reduces drift on long runs.

Compaction, done in tiers#

The naive approach — summarize the transcript when it gets long — loses exactly the things you need: identifiers, file paths, error strings, numbers. A tiered scheme keeps the recoverable parts recoverable.

class TieredMemory:
    """Recent turns verbatim, older turns compressed, facts extracted and kept.

    The invariant: anything with a stable identifier survives compaction, because
    identifiers are what let the agent re-fetch detail it discarded.
    """

    def __init__(self, verbatim_turns=6, budget=24_000):
        self.verbatim = deque(maxlen=verbatim_turns)   # tier 1: last N turns
        self.compressed: list[str] = []                # tier 2: summarized spans
        self.facts: dict[str, str] = {}                # tier 3: extracted, pinned
        self.budget = budget

    def append(self, turn):
        if len(self.verbatim) == self.verbatim.maxlen:
            evicted = self.verbatim[0]
            self.facts.update(extract_identifiers(evicted))   # ids, paths, versions
            self.compressed.append(compress(evicted))
        self.verbatim.append(turn)
        self._enforce()

    def _enforce(self):
        while self.tokens() > self.budget and len(self.compressed) > 1:
            # fold the two oldest summaries into one — lossy, but bounded
            a, b = self.compressed.pop(0), self.compressed.pop(0)
            self.compressed.insert(0, compress_pair(a, b))

    def render(self) -> str:
        return "\n\n".join([
            f"## Established facts\n{fmt(self.facts)}",
            f"## Earlier (summarized)\n{chr(10).join(self.compressed)}",
            f"## Recent\n{fmt_turns(self.verbatim)}",
        ])

extract_identifiers is the load-bearing function. A summary that says "explored the authentication module" is nearly useless; one that says "explored src/auth/session.ts, found validateToken at line 88, which throws TokenExpiredError" lets the agent go back. Identifiers are pointers, and pointers survive compression at almost no cost.

Tool results are the biggest offender#

A single directory listing, database query, or file read can be tens of thousands of tokens, and it is usually 95% irrelevant to the decision being made.

Three rules, applied at the tool boundary rather than in the prompt:

That third one is where most of the savings live in long runs, and almost no framework does it by default.

Fleet-level context#

With multiple agents, context becomes a distributed systems problem, and the question is what crosses the boundary.

Sub-agent to parent: conclusions only. A sub-agent that returns its transcript has defeated the purpose of being a sub-agent. Enforce this with a schema — the return type is a structured result object, not a string — and the temptation disappears.

class SubAgentResult(BaseModel):
    finding: str = Field(max_length=1200)     # what the parent needs to know
    confidence: Literal["high", "medium", "low"]
    sources: list[str]                        # ids, so the parent can re-fetch
    unresolved: list[str] = []                # what it could not determine

The unresolved field earns its place: the most common multi-agent failure is a sub-agent quietly not doing part of its job and returning a confident summary of the part it did.

Shared state: explicit, versioned, small. If agents genuinely need to see each other's work, put it in a store with an interface, not in a shared prompt. Versioned, attributable writes; agents read what they query for. This makes the "who changed this" question answerable, which it is not when the shared state is a growing string.

Nothing implicit crosses. Every piece of shared context should be traceable to a specific call. Ambient state in a multi-agent system is where the non-reproducible bugs live.

Failure modes and their signatures#

Thrashing. The agent repeatedly re-fetches material it already had, because the compaction evicted the content and did not keep the identifier. Signature: duplicate tool calls with identical arguments. Instrument this — it is a one-line check and a reliable alarm on a broken memory policy.

Instruction drift. Around step forty, the agent starts optimizing for something adjacent to the task. Signature: outputs that are locally sensible and globally off-target. Fix: pin the task at the tail, and periodically re-inject an explicit "the goal is still X" turn.

Poisoned memory. An early wrong conclusion gets extracted into the facts tier and is thereafter treated as ground truth, with the evidence long since evicted. This is the nastiest one, because compaction has removed the ability to re-litigate it. Mitigation: keep provenance on every extracted fact, and mark facts derived from inference rather than observation as revisable.

Silent truncation. The context exceeds the window, the framework drops the middle, and nothing tells you. Fix: count tokens before every call and fail loudly rather than truncating implicitly.

Instrumentation#

Four numbers, on a dashboard, per run:

• Token utilization by region. If tool definitions are 30% of your window, that is a finding.
• Compaction ratio over time. Rising ratios mean the agent is losing more detail per step; a run that ends at 40:1 compression has effectively forgotten its beginning.
• Re-fetch rate. Duplicate tool calls per run. The thrashing alarm.
• Steps to completion. Distribution, not mean. A long tail is where context degradation shows up before it shows up in success rate.

The short version#

Treat the window as a managed resource. Pin what must never be lost. Load tools by phase. Truncate at the tool boundary and return handles rather than payloads. Compact in tiers and always keep identifiers. Cross agent boundaries with typed conclusions, never transcripts. Measure utilization, compaction, and re-fetch.

None of this is exotic. It is memory management, and it is the difference between an agent that works for ten steps and one that works for a hundred.