What happens when the agent conversation fills the window?

When the conversation approaches the window limit, the harness makes a separate model call that turns the older turns into a summary, replaces those turns with it, keeps the most recent ones verbatim, and carries on. The session neither dies nor restarts. It loses detail, quietly, and continues with a history rewritten by a model.

This is the last resort of the harness’s context management: it only fires once dropping old tool output stops freeing enough room. The technique itself is the same context compaction and summarisation used outside agents. What changes inside an agent is the frequency and the stakes: it isn’t a conversation going vague, it’s a task changing direction.

The side effect starts earlier than that, incidentally. The model already drifts in long conversations with no compaction involved at all, and compaction stacks a second source of loss on top of the first.

What happens, step by step

1. The harness crosses a threshold. Around 90% occupancy, depending on configuration, with room reserved for the answer.
2. It picks the slice to compact. Almost always everything except the two or three most recent turns, which stay verbatim.
3. It assembles a summary prompt. A fixed prompt, written by whoever built the tool, stating what to preserve. This is where most of the quality is decided, and you rarely get to see it.
4. It calls the model. 150K tokens in, a few thousand out. That call costs more than any ordinary iteration of the task.
5. It substitutes. The old slice leaves, the summary takes its place, recent turns stay where they were.
6. It re-pastes what must stay literal. A careful harness reinjects certain items without routing them through the summary. That’s the bottom box in Figure 1.
7. It carries on. The model receives the next request with no way of knowing a cut happened, unless the harness says so in the context.

Step 3 deserves attention because it’s the only one adjustable from outside, when the tool allows it. A decent summary prompt asks for four things by name: the original goal, the decisions along with the reason for each, the constraints in literal text, and the current state of the work, including what has been verified and what hasn’t. A bad one asks for “a summary of the conversation”, and the model delivers exactly that: a narrative of what was discussed, which is the least useful possible shape for resuming a task.

The canonical formulation of this idea treats the context window as main memory and external storage as secondary memory, with the model itself issuing function calls to move data between the two tiers and interrupt the flow when needed¹. The distinction is worth keeping: that proposal hands the model control over what leaves and what comes back. The automatic compaction most tools perform is simpler — the harness decides, and the model only writes the summary.

What should never enter the summary

A decision already made. “We’re going with Postgres” has to arrive on the other side as a decision, not as “there was a discussion about databases”. The second form invites the agent to reopen the argument, and reopening costs iterations.

A constraint the user gave. “Don’t touch migrations/” has the shape of a chat message and the weight of a system rule. A summary that turns it into “the user asked for care around migrations” removed the part that mattered, which was the exact path.

File paths, line numbers, function names. They are the only way back to the right place. Summarisation tends to replace them with description: “the payment service file” instead of src/billing/charge.py:212. A description doesn’t open in the editor.

Identifiers. Commit hashes, issue numbers, order ids, error keys. One wrong digit is worse than absence, because the agent acts confidently on the wrong value.

What the four share is their literal nature. Summarisation is a semantic operation, and what it preserves well is meaning, not the character string. So the fix isn’t writing a better summary prompt: it’s keeping these items out of the summary’s path altogether.

There are three ways to do that, from flimsiest to most reliable. The first asks the summary prompt to preserve paths and identifiers, and it fails precisely when the history is large, which is when you need it. The middle one has the harness keep a list of pinned items and re-paste them after the summary, never routing them through the model; that’s what the bottom box in Figure 1 represents, and it works as long as somebody marked the items. The strongest writes outside the window, into a file the agent re-reads. Only the third survives the whole session going down.

Recursive compaction

In a long session, compaction runs more than once, and the second pass receives the first pass’s output as input.

The loss isn’t linear. Whatever survives one pass is already the most generic material that was there, so the next pass works on content with less detail to lose and loses some anyway. Over three passes, “use Postgres over MySQL because of the licence” arrives as “backend decided”, which is true and useless.

Summarising recursively isn’t an implementation flaw; it’s a research proposal. A 2023 paper generated dialogue memory recursively, combining the previous memory with the next stretch of conversation, and reported more consistent responses in long conversations across both open and closed models². What that work measured was response consistency in dialogue. Preserving a file path or the reason behind an architecture decision across eighty coding iterations is a different problem, and not the one it evaluated.

Why the summary loses what it loses

Automatic summarisation has a failure mode that was known and measured long before agents existed. A large-scale human evaluation published in 2020 analysed several neural abstractive summarisation systems and found substantial amounts of hallucinated content in all of them, meaning statements absent from the source document³. Those models predate current LLMs and the rates have aged; the failure taxonomy hasn’t. A summary can add, not only omit.

Worse: detecting that is hard for the model itself. A 2023 benchmark tested LLMs on judging whether a summary is factually consistent with its source, across ten domains, and most models landed close to chance; the best of them still came in 8% below estimated human performance⁴. The implication for the harness is direct: you can’t close the loop by asking the model to check its own summary. It isn’t good at that.

What long-memory benchmarks show

There’s a way to measure the effect without looking at the summary. A 2024 benchmark evaluated chat assistants on five long-term memory abilities (information extraction, multi-session reasoning, temporal reasoning, knowledge updates and abstention), using 500 questions embedded in long histories, and reported a 30% accuracy drop for commercial assistants and long-context models when they had to retain information across sustained interactions⁵.

The number comes from one specific arrangement of questions and isn’t a compaction loss rate. It’s useful for calibrating expectations: keeping information alive across a long session is an open problem, not a configuration detail.

The equivalent measurement on your own work is cheap. Take five sessions that compacted, write down the constraints and decisions stated before the cut, and check which ones the agent still honoured afterwards. Five sessions is enough to tell whether your problem is compaction or something else entirely, and it’s the only evidence that speaks about your tasks.

Where compaction breaks

The typical case has this shape. You ask for an adjustment to the shipping calculation and mention, in the second message, that legacy/shipping_v1.py is frozen and must not be touched. Thirty-odd iterations later, the agent compacts. The summary records “the user asked for care around the legacy shipping code”. At iteration 40 the most direct fix runs straight through that file, and the agent edits it, because what survived in context was advice rather than a prohibition. The permission layer stops and asks, you approve without remembering what you agreed thirty iterations ago, and nobody in the loop held the whole picture.

The loss produces no event. No log line says “the user’s constraint left”. The symptom shows up as an action contradicting something agreed, and by then the record to compare against is gone.

The summary prompt is generic. It was written for every task, so it has no idea that in a database migration project the word “licence” is the central fact.

The agent may compact at the worst moment. The window fills fastest when the task is most complex, which means the cut lands at the point of highest information density.

Compacting doesn’t refund the cost. The summary call is paid for with the whole history as input. Compacting 150K tokens costs more than several ordinary iterations, and a session that compacts three times spent half a million tokens on that alone.

A long session isn’t always the goal. Much of what compaction exists to rescue should never have entered the session. Two independent tasks in the same conversation compete for the same window with nothing gained.

It throws away the prompt cache. Compaction rewrites the middle of the prompt, so every cached prefix that came after that point stops matching. The call right after a compaction pays full price on a context you had been getting at a discount for twenty iterations, which is part of why the bill jumps in a place that doesn’t look related to the summary.

What to do

1. Write the state to a file before you need to. Decisions, constraints, paths and next steps. It’s the only copy that doesn’t depend on a summary having chosen to keep it.
2. Force compaction at a moment you pick. Compacting at 60% with the task in a stable state beats compacting at 92% mid-investigation.
3. Open a new session per task. The most effective measure and the least used. Session resume handles continuity without dragging the whole history along.
4. Truncate tool output at the source. Most of what compaction has to swallow is command output that should never have entered whole.
5. Read the summary when it happens. If the tool shows it, read it. Thirty seconds there saves the iterations the agent would spend redoing what was left out.
6. After compacting, restate the constraints. One line. It’s cheap and it fixes the most common failure mode.

Step 1 is the one that changes the game, and it doesn’t depend on your harness being any good. A notes file survives compaction, a dropped session and a change of tool, because the harness isn’t the thing holding it.

What it costs

Order of magnitude, to get the shape of the bill. One compaction in a 200K-token window processes somewhere between 120K and 180K input tokens and produces 3K to 8K of output. That’s comparable to five or ten ordinary iterations of the same task, concentrated into a single call.

A session that compacts three times spends on compaction roughly what twenty iterations of actual work would cost. The comparison that matters isn’t the absolute price: it’s against opening a fresh session with a 2K-token notes file, which does the same job for a fraction of the cost and without the semantic loss.

Footnotes

1. Packer et al. (2023) proposed virtual context management inspired by the memory hierarchy of operating systems, with the model moving data between tiers through function calls and interrupts. ↩

2. Wang et al. (2023) generated dialogue memory recursively, combining the previous memory with the next stretch of conversation, and reported more consistent responses in long conversations across open and closed models. ↩

3. Maynez et al. (2020) ran a large-scale human evaluation of neural abstractive summarisation systems and found substantial amounts of hallucinated content in all generated summaries. ↩

4. Laban et al. (2023) built a ten-domain benchmark for detecting factual inconsistency in summaries; most models performed close to chance and the best one still came in 8% below estimated human performance. ↩

5. Wu et al. (2024) evaluated five long-term memory abilities with 500 questions embedded in long histories and reported a 30% accuracy drop for commercial assistants and long-context models. ↩