Context Compaction

Long-running agentic loops eventually run out of context window: every tool result and every reasoning block stays in history forever unless something trims it. compaction is an opt-in call option that does this automatically, in three cheapest-first layers, without you writing a token counter or a pruning function yourself.

What's first-class here

Context management is often left to you: estimate tokens, then prune history inside a per-step hook. compaction makes that automatic and provides the pieces you'd otherwise hand-roll:

It stays a manual escape hatch too: prepareStep runs after compaction each step, so you can always rewrite the already-compacted history yourself and get the final word.

Turning it on

compaction lives on CommonCallOptions, so it works identically in generateText and streamChat. It only ever runs inside the agentic loop — a call with no tools (a single-turn call) never invokes it, regardless of the option.

compaction?: 'auto' | CompactionPolicy;

interface CompactionPolicy {
  threshold?: number;             // default 0.92 — context-fill ratio that triggers compaction
  keepRecentSteps?: number;       // default 4 — most-recent assistant turns, always protected
  layers?: CompactionLayer[];     // default ['prune-tool-results', 'prune-reasoning', 'summarize']
  summarizeModel?: LanguageModel; // default: the loop's own model
  countTokens?: (messages: Message[]) => number; // 2.0 — a real tokenizer
}

type CompactionLayer = 'prune-tool-results' | 'prune-reasoning' | 'summarize';
type CompactionOption = 'auto' | CompactionPolicy;

'auto' is shorthand for {} (every field defaulted). It's off by default — omit compaction and history grows unbounded, same as today.

import { generateText } from '@deuz-sdk/core';
import { createAnthropic } from '@deuz-sdk/core/anthropic';

const anthropic = createAnthropic({ apiKey: process.env.ANTHROPIC_API_KEY! });

const result = await generateText({
  model: anthropic('claude-opus-4-8'),
  messages: longRunningHistory,
  maxSteps: 30,
  tools: { /* … */ },
  compaction: 'auto', // trigger at 92% fill: prune tool results → prune reasoning → summarize
});

A tuned policy — skip the reasoning layer, summarize with a cheaper model:

const result = await generateText({
  model: anthropic('claude-opus-4-8'),
  messages: longRunningHistory,
  maxSteps: 30,
  tools: { /* … */ },
  compaction: {
    threshold: 0.85,
    keepRecentSteps: 6,
    layers: ['prune-tool-results', 'summarize'], // no prune-reasoning
    summarizeModel: anthropic('claude-haiku-5'),
  },
});

The three layers

Layers run in policy order against messages the invariants below don't protect, re-estimating fill after each one, stopping as soon as fill drops under threshold * 0.8:

1. prune-tool-results — old tool_result bodies become a [pruned N chars] stub. toolUseId and isError are kept untouched, so the Anthropic "every tool_use needs a tool_result" invariant still holds.
2. prune-reasoning — old assistant reasoning parts are dropped. The last assistant turn is never touched by this layer, protecting the extended-thinking signature chain.
3. summarize — the oldest unprotected contiguous run of messages collapses into one new user-role summary message. This costs one extra model call (using summarizeModel); that call's usage counts toward result.usage and toward budget stops (see below).

What's always protected

Regardless of policy, no layer ever touches:

• every system message,
• the first user message,
• the last message in history (the pending question — critical when no assistant turn exists yet),
• the last keepRecentSteps assistant turns and everything after them.

Claude 5.x: simple compaction (2.3)

Claude Fable 5.1, Opus 5.5, Sonnet 5.5 and Haiku 5.5 bind every thinking block to the conversation before it: the top-level system, the tools array and every earlier message have to reach the API byte-identical, or the replayed block is invalid (Preserved thinking). The layers above edit exactly that prefix: summarize replaces the oldest turns and then replays the recent turns with their thinking behind the new summary, and the prune layers rewrite messages in the middle of it.

So when the model being compacted for is one of these (or an unknown claude-*-5… id on the family fallback), a pass runs simple compaction instead:

• One summary replaces the whole conversation. No assistant turn survives the boundary, so no thinking from before it is ever replayed with a changed prefix. The summary is the same rolling user message ([Earlier conversation summarized]), folded into on the next pass.
• What stays: the leading system messages (the top-level system, untouched), every later system message that carries run state (handoff instructions, tool-set changes, your own), in order, and the pending user turn after the last assistant message together with its memory-recall message. An older recall block is conversation content: it is summarized with the turns around it. A history that ends mid tool loop (on a tool result) is summarized whole.
• The prune layers never run on their own. The summary is the one declared boundary at which the prefix may change. With 'auto' (or any policy that lists summarize) they simply fold into it; a policy that lists only prune layers compacts nothing on these models and reports each skipped layer through the usual skip warning.
• Overflow recovery follows the same rule: a provider rejection forces one simple pass and a retry.

The other models keep the three layers exactly as described above. Anthropic's server-side compaction (the beta compaction request parameter and its signed compaction blocks) is not used: it needs a new canonical part kind, which 2.x does not add. Not live-verified.

Invariants

• Immutable, cache-stable history. Compaction builds new arrays; it never mutates a prior message, and untouched messages keep reference equality so provider prompt-cache hits survive. response.messages — what the loop appended — is never affected by compaction; it only ever reflects what happened, not a compacted rewrite. The summarize layer does break the cache once (a new message replaces a run of old ones), but the history re-stabilizes from that point on.
• Never throws. A failing summarize call (model error, bad output) logs a warning and skips just that layer — it does not fail the call or kill the loop. prune-tool-results and prune-reasoning cannot fail (they're pure string/array transforms).
• Token counts are a calibrated heuristic. There is no tokenizer and no network call spent counting — fill is estimated, and a session-local EMA factor tightens the estimate against each step's real provider-reported usage as the loop runs. Treat threshold as approximate, not exact.

Streaming: the compaction part

streamChat emits one compaction StreamPart (and Deuz UI wire part) per layer that ran and actually changed the history, right before the step it applies to:

interface CompactionPart {
  type: 'compaction';
  layer: 'prune-tool-results' | 'prune-reasoning' | 'summarize';
  tokensBefore: number;
  tokensAfter: number;
  /** 2.0 additive; absent = 'threshold', the only trigger before 2.0. */
  trigger?: 'threshold' | 'manual' | 'overflow';
}

for await (const part of result.fullStream) {
  if (part.type === 'compaction') {
    console.log(`compaction: ${part.layer} ${part.tokensBefore} → ${part.tokensAfter}`);
  }
}

generateText has no stream to emit on — it logs the same event via deps.logger.info instead, one line per layer that ran.

Budget stops see compacted-in summarize usage too

summarize's one extra model call is real, metered usage — it's included in result.usage, so it also counts toward totalTokensExceed / costExceeds if you've set one. A long research loop that leans hard on compaction: 'auto' should budget for the occasional summarize call.

Interop with Anthropic's native context editing

Anthropic's server-side context editing (providerOptions.anthropic.context_management) is untouched by this feature and still works exactly as before — it's a completely separate mechanism (Anthropic edits the conversation on their side, not ours). Its applied_edits report comes back on the finish part's providerMetadata.anthropic.contextManagement. You can use either, or both: compaction acts on the canonical Message[] history at the SDK layer regardless of provider, while context_management is Anthropic-only and opaque to the SDK.

Beyond the automatic policy

2.0 adds three things this page does not cover, all on Compaction & context recovery:

• compactMessages() — the same layers as a plain function over a Message[] you own. Pure by default: with no summarizer it makes no network call at all.
• The rolling summary — pass N folds into pass N−1's summary instead of restarting, so at most one summary block ever sits in the history.
• Overflow auto-recovery — a provider that rejects a request as too long triggers a forced compaction pass and one retry, even when compaction was never set. It fires on every chat wire whose adapter recognizes the rejection: anthropic, chat_completions, and responses (which reuses the chat_completions error mapping). The Gemini native wire does not map it.

See also

• Compaction & context recovery — compactMessages, the rolling summary, overflow recovery, countTokens.
• Tools — prepareStep (compaction runs before it, every step) and budget stop conditions.
• Tool loop — the agentic loop this feature only activates inside.
• Sub-agents — agentTool accepts its own compaction policy for long-lived research agents.
• streamChat — the fullStream this feature's compaction parts ride on.