Summarisation in Agent Loops: Compressing Without Forgetting

Every long agent session eventually runs into the same wall. The transcript grows with each tool result, the window fills, and either the request fails outright or performance degrades well before it does.

Summarisation is how you keep going. It is also the operation most likely to silently break an agent, because a summary that drops one constraint produces an agent that confidently violates it for the rest of the run.

Why compress at all

Three reasons, and only one of them is the window limit.

Cost is the first. The full transcript is resent on every turn, so a bloated context is paid for repeatedly, not once. Shedding twenty thousand tokens on turn fifteen saves them on every remaining turn. Agent token budgets covers the arithmetic.

Attention is the second and the more important one. Even well inside a large window, material in the middle of a long context is attended to less reliably than material at either end. A shorter, denser context is not just cheaper, it is followed more closely. The lost-in-the-middle problem covers the effect.

What must never be summarised

Draw this boundary before you write any compaction code, because it is the part that causes incidents.

The system prompt is untouchable. Any constraint that must hold for the whole session belongs there specifically because it is never compacted — never in a conversational message that a later pass can decide is old news.

The current task statement is untouchable. So is anything the user said that constrains the output: a required format, a forbidden library, a deadline, an explicit "do not touch the migrations directory". These have a habit of being phrased conversationally and then summarised into oblivion.

The last few turns stay verbatim. The model needs exact detail about the immediate situation, and a paraphrase of the error message it is currently debugging is worse than useless.

Three tiers, not two

The structure that works in practice has three layers rather than a simple old-versus-new split.

Pinned: system prompt, task, hard constraints. Always present, never rewritten, positioned where attention is strong.

Summarised: the middle of the session, compressed into a structured account of what has been established. This is where the compression ratio comes from, because it is where the bulk of superseded tool output lives.

Verbatim: the most recent turns in full. Enough to preserve working detail, usually a handful of turns rather than a fixed token count. Context compaction strategies covers the alternatives to summarising, such as stubbing individual tool results.

What a good summary contains

Not prose. A summary written as narrative reads well and loses the specifics the agent needs. Use a fixed template and fill the same fields every time.

Five fields cover nearly everything: what has been established as fact, with file paths and identifiers intact; what has been decided and why; what has been tried and failed, stated concretely enough that it will not be retried; what is still open; and what the immediate next step was.

The failed-attempts field is the one teams omit and the one that pays for itself fastest. Without it, a compacted agent cheerfully retraces its own dead ends, which is both the most expensive and the most demoralising failure to watch in a trace.

When to trigger it

Threshold-based triggering — compact when the transcript passes some share of the usable window — is the sensible default. It adapts to sessions that fill quickly and leaves short sessions alone.

Avoid compacting on a fixed turn count. Turns vary enormously in size; twenty small ones may need nothing while three large file reads may need it immediately.

Be aware that compaction destroys your prompt cache. Everything after the rewritten portion is a cache miss on the next turn, so a system that compacts constantly can spend more on lost cache hits than it saves in context. Compact in fewer, larger steps rather than continuously. Prompt caching covers why prefix stability matters.

Who writes the summary

Using a cheaper model for compaction is attractive and works, provided the template is strict. The task is extraction against a fixed schema, not judgement, and small models do that acceptably.

The risk is that a weaker model discards something load-bearing because it did not recognise its importance. Mitigate structurally rather than by hoping: pass the pinned constraints into the summarisation prompt as things that must be preserved verbatim, and have the harness re-attach them afterwards regardless of what came back.

Never let the summariser rewrite a previous summary in free-form. Repeated summarisation of summaries drifts, and after three rounds the account of what happened bears a decreasing relationship to what actually happened. Summarise the new material and append it to the structured fields instead.

Testing that it did not lose anything

Compaction quality is measurable, and almost nobody measures it. The straightforward test is a set of questions whose answers were in the pre-compaction context: which files have been modified, what did the last test run report, what was the user's formatting requirement.

Ask them of the compacted context and check the answers. A summariser that fails these on your own sessions will fail them in production, and you will see it as inexplicable agent behaviour rather than as a compaction bug.

Watch the compression ratio too. A summariser producing a summary nearly as long as its input is not helping, and one producing three lines from thirty thousand tokens has certainly dropped something. Both are visible from a single log line per compaction. Agent memory and context management covers the wider picture, and a durable scratchpad is a good place to keep findings that must survive every compaction untouched.