An agent that answers one question never has a context problem. An agent that spends four hours refactoring a codebase has nothing but context problems. Every tool call adds output, every file read adds content, every reasoning step adds tokens, and the window — however large — eventually fills. What happens at that boundary determines whether your agent degrades gracefully or falls over.
Compaction is the mechanism that handles it. When the conversation approaches a threshold, earlier history gets replaced by a summary of itself, freeing room while preserving the thread. The session continues. The details, some of them, do not.
What changed in 2026
- Compaction moved server-side. It used to be something you implemented yourself — track token counts, call a summarisation prompt, splice the result back into the history. Providers now offer it as a request parameter, which removes the fiddly part and adds a subtle new failure mode.
- Thresholds became configurable rather than fixed. Triggering at a set token count lets you tune when the quality cost of summarising is worth paying, instead of accepting a vendor default.
- The distinction from context editing hardened. Two different operations, frequently conflated: one summarises, one deletes. Teams that use both deliberately get considerably better results than teams that reach for whichever they heard of first.
- Million-token windows did not make it obsolete. They raised the ceiling and made long sessions economically painful well before they became technically impossible. Compaction is now as much a cost control as a capacity one.
Compaction vs context editing
|
Compaction |
Context editing |
| Operation |
Summarises earlier turns |
Clears specific blocks outright |
| What survives |
A condensed account of everything |
Everything except what you targeted |
| Best for |
Long conversational or reasoning history |
Bulk tool results that are no longer relevant |
| Loses |
Detail and specificity |
The cleared content entirely, by design |
| Cost |
A model call to produce the summary |
Effectively free |
| Reversible |
No |
No |
The rule of thumb worth internalising: if the content had a purpose that is now finished — a file you already read and acted on, a search result you already used — clear it. If it carries reasoning or decisions the agent still needs to honour, compact it. Deleting decisions produces an agent that contradicts itself; summarising bulk output wastes a model call to compress something you should have thrown away.
Wiring it up without losing state
The single most common implementation bug is subtle enough to survive code review. When compaction is active, the response comes back carrying compaction blocks alongside the usual text. Those blocks are how the provider represents the compressed history, and they must go back in your next request.
If your code does something like messages.push({role: "assistant", content: response.text}) — extracting just the string, which is the natural thing to write — you have discarded the compaction state. Nothing errors. The conversation continues. The context simply grows as though compaction never ran, and you find out when you hit the wall you were trying to avoid.
Append the full content array instead. The same discipline applies to thinking blocks and tool-use blocks; treating a model response as "the text it produced" rather than "the structured content it returned" is a recurring source of quiet breakage. The observability practices in LLM observability will surface it faster than reading code will — a token count that keeps climbing past your threshold is the giveaway.
What compaction reliably loses
Summarisation compresses toward meaning, which means it discards precision first. In practice the casualties are consistent:
Exact identifiers. File paths, ticket numbers, commit hashes, variable names. A summary says "updated the authentication middleware"; it does not say src/middleware/auth.ts:142.
Numbers. Specific values get rounded, generalised, or dropped. If the agent measured something in turn 12 and needs the figure in turn 90, it may confidently recall a different one.
Negative constraints. "Do not touch the billing module" is easy to summarise away, because summaries prioritise what happened over what was prohibited. This one causes real damage.
Ordering. The sequence of events compresses into a narrative, and narratives smooth over the order in which things actually occurred.
The defence is to keep load-bearing facts outside the conversation entirely. A scratchpad file, a memory store, or a system-level instruction survives compaction because it is not part of the compacted history. Anything the agent must not forget belongs there, not in a message you hope the summariser respects. This is the same argument that motivates context engineering generally: what you put in the window matters less than what you keep out of it.
Common mistakes
- Appending only the text of the response. The compaction blocks vanish and compaction silently stops working. Append full content.
- Compacting too early. Every compaction costs a model call and loses detail. Triggering at 20% window utilisation burns both for no benefit.
- Compacting too late. Waiting until you are one message from the ceiling means the summarisation itself may not fit. Leave headroom.
- Assuming the summary is faithful. It is a model output, with the usual properties of model outputs. Critical constraints need to live somewhere else.
- Using compaction where editing belongs. Paying a summarisation call to compress a 40KB tool result you will never reference again is pure waste.
- Never testing a compacted session. Most agents are tested on short interactions and deployed on long ones. Run a session past the threshold deliberately and check what the agent still knows.
FAQ
Does compaction reduce my token bill?
Indirectly, and it depends. You pay for the summarisation call, then pay less per subsequent request because the history is smaller. On a long session that is a clear win; on a medium one it can be a wash. Combine it with prompt caching carefully — compaction rewrites the prefix, which invalidates the cache at the moment it fires.
How is this different from an agent's memory?
Compaction is about fitting a single session into a window. Memory is about carrying knowledge across sessions. They solve adjacent problems and are often used together — AI agent memory covers the persistent side.
Can I control what the summary preserves?
Some implementations let you steer it with an instruction. Where available it is worth using, and the highest-value instruction is usually to preserve constraints and identifiers verbatim rather than to summarise them.
What happens if compaction fails mid-session?
You get an error or a truncated context, depending on the provider. Long-running agents should treat it as a recoverable failure — checkpoint enough state outside the conversation that you can restart the session rather than lose the work.
Where to go next
If you are building agents that run long enough to need this, AI agent observability covers the traces that make compaction behaviour visible, and agent token budgets covers pacing the spend that long sessions generate. For the upstream question of what should be in the window in the first place, start with context engineering.