The context window is the model’s working memory, and it holds everything in a request: the system prompt, every message including tool results, images and documents, the tool definitions, and the output the model generates for the turn, thinking included. Capacity is not the whole story. Anthropic’s docs state that more context is not automatically better: as token count grows, accuracy and recall degrade (“context rot”), so curating what is in context matters as much as how much fits. Window sizes are per model (the September 2026 Models overview lists 1M tokens for Fable 5.1, Opus 5 and Sonnet 5 and 200K for Haiku 4.5), so check the current page rather than relying on memory.
Budget the window like a resource
Break the budget into four parts: fixed (system prompt, tool definitions), growing (conversation history, tool results, retained thinking blocks), per-request (retrieved documents, user input) and reserved (max_tokens for output, which includes thinking). Instrument it rather than guessing:
- Every response reports
usage. The token counting endpoint estimates a request before you send it, is free, and has its own rate limits; it returns an estimate that can differ slightly from billed tokens. - Count against the model you will run. Claude 4.7 and later models use a newer tokenizer that produces roughly 30 percent more tokens for the same text, so counts from an older model must not be reused for cost or fit.
- Prompt caching changes what you pay for cached tokens, not whether they count: cached prefixes still occupy the window.
- Overflow behaves differently by case: if input alone exceeds the window the API returns a 400 error; on Claude 4.5 and newer, input plus
max_tokensbeyond the window is accepted and generation can end withstop_reason: model_context_window_exceeded, so check the stop reason.
Order and shape what goes in
Anthropic’s long-context guidance for inputs of 20k tokens and up: put long documents at the top, above the query, instructions and examples; put the query at the end (the docs report up to a 30 percent quality gain in their tests with complex multi-document inputs, so treat it as directional and test on your data); wrap each document in tags with source metadata; and for long-document tasks ask the model to quote the relevant passages first, then answer from those quotes. Position effects are a reason to restructure the input, not to add a reminder to “pay attention to everything.”
Then reduce what enters the window at the source:
- Trim tool results. An order lookup that returns dozens of fields when five matter will re-consume those tokens on every later turn. Filter inside the tool implementation or in the harness code that wraps the tool call, before the result enters history.
- Make upstream agents return structured findings (claim, source, date) instead of transcripts and reasoning (see 2.4).
- Load just in time. Keep lightweight identifiers and fetch data when needed, and avoid loading every tool definition up front (see 1.8 and the tool search option in the docs).
- Remember the API is stateless. Each request carries the history you choose to send; whatever you drop is gone.
Manage growth: clear, compact or externalise
Anthropic documents server-side mechanisms, both in beta at the time of writing, so confirm current names, headers and supported models in the docs:
| Mechanism | What it does | Watch out for |
|---|---|---|
Tool result clearing (clear_tool_uses_20250919, context editing) | Clears older tool results once input passes a trigger (default 100,000 input tokens), keeping the most recent tool uses (default 3). Options include clear_at_least, exclude_tools and clear_tool_inputs. | The model loses that data and clearing invalidates the cached prefix; use clear_at_least so a clear frees enough to justify it, and pair with the memory tool to save what matters first. |
Thinking block clearing (clear_thinking_20251015) | Controls how many prior thinking turns are kept. Newer models keep prior thinking by default, and kept blocks count as input. | Keeping blocks preserves cache hits; clearing frees window but invalidates the cache at that point. List it before tool clearing when combined. |
Server-side compaction (compact_20260112) | Summarises the conversation when input reaches a trigger (default 150,000, minimum 50,000) and returns a compaction block that replaces everything before it. The context windows page calls it the primary strategy for long-running conversations and agentic workflows. | You must pass the compaction block back; a custom instructions string replaces the default summary prompt entirely; compaction is an extra sampling step, so sum usage.iterations for true cost; check the model support list (Haiku 4.5 is not on it as of this writing). |
| External notes and sub-agents | Persist state outside the window (memory tool, files) or delegate work to sub-agents that return condensed summaries. | Needs a retrieval or hand-off design; see 2.4. |
Some models also have context awareness: Sonnet 5, Sonnet 4.6, Sonnet 4.5 and Haiku 4.5 track their remaining budget through API-injected tags, which can make them wrap up early near the limit unless your prompt tells them compaction or external state is in play.
Key concept: reduce by information loss, cheapest first
A workable ordering is to prevent tokens entering (trim at the source, load just in time), then remove what is provably dispensable (clear stale tool results), then summarise (compaction), and finally move state outside the window. Each step is lossier than the one before, so apply the least lossy step that solves the problem, and protect critical facts explicitly at every step.
The summarisation trap and long-context failure modes
Summaries compress, and compression drops precisely the details a transactional system needs: amounts, dates, identifiers, promised deadlines. “Customer wants a refund for a recent order” has lost the amount, the order number and the date. The fix is structural. Keep a persistent facts block (a small structured record of the exact values) that you maintain and re-inject on every turn outside the history that gets summarised, and when you use compaction supply custom instructions that name what must be preserved verbatim. Then test it: after a compaction, ask for the exact values and check them.
Other failure modes: stale or duplicated tool output accumulating turn after turn; early errors that persist and get baked into summaries; critical instructions buried mid-context; and long-horizon agents that lose state at a window boundary because nothing was written outside it. Big windows delay these problems, and add cost and latency on every call.
Common exam distractor
“Use a larger context window so nothing needs to be trimmed or summarised” and “tell the model to preserve numbers when summarising” are tempting but wrong. A bigger window postpones the limit while context rot, cost and latency still grow, and an instruction to a summariser is probabilistic. The robust answer is a persistent facts block, targeted trimming or clearing, and a test that verifies the critical values survive.