Study guides / CCAR-P / Domain 6

Claude Models, Prompting & Context Engineering · Lesson 4 of 5

6.4 - Optimising the Context Window and Managing Token Usage

Budget the context window deliberately: decide what to load, in what order, and when to trim, clear, compact or externalise, and recognise the failure modes of long contexts.

The context window is the model’s working memory, and it holds everything in a request: the system prompt, every message including tool results, images and documents, the tool definitions, and the output the model generates for the turn, thinking included. Capacity is not the whole story. Anthropic’s docs state that more context is not automatically better: as token count grows, accuracy and recall degrade (“context rot”), so curating what is in context matters as much as how much fits. Window sizes are per model (the September 2026 Models overview lists 1M tokens for Fable 5.1, Opus 5 and Sonnet 5 and 200K for Haiku 4.5), so check the current page rather than relying on memory.

Budget the window like a resource

Break the budget into four parts: fixed (system prompt, tool definitions), growing (conversation history, tool results, retained thinking blocks), per-request (retrieved documents, user input) and reserved (max_tokens for output, which includes thinking). Instrument it rather than guessing:

Order and shape what goes in

Anthropic’s long-context guidance for inputs of 20k tokens and up: put long documents at the top, above the query, instructions and examples; put the query at the end (the docs report up to a 30 percent quality gain in their tests with complex multi-document inputs, so treat it as directional and test on your data); wrap each document in tags with source metadata; and for long-document tasks ask the model to quote the relevant passages first, then answer from those quotes. Position effects are a reason to restructure the input, not to add a reminder to “pay attention to everything.”

Then reduce what enters the window at the source:

Manage growth: clear, compact or externalise

Anthropic documents server-side mechanisms, both in beta at the time of writing, so confirm current names, headers and supported models in the docs:

MechanismWhat it doesWatch out for
Tool result clearing (clear_tool_uses_20250919, context editing)Clears older tool results once input passes a trigger (default 100,000 input tokens), keeping the most recent tool uses (default 3). Options include clear_at_least, exclude_tools and clear_tool_inputs.The model loses that data and clearing invalidates the cached prefix; use clear_at_least so a clear frees enough to justify it, and pair with the memory tool to save what matters first.
Thinking block clearing (clear_thinking_20251015)Controls how many prior thinking turns are kept. Newer models keep prior thinking by default, and kept blocks count as input.Keeping blocks preserves cache hits; clearing frees window but invalidates the cache at that point. List it before tool clearing when combined.
Server-side compaction (compact_20260112)Summarises the conversation when input reaches a trigger (default 150,000, minimum 50,000) and returns a compaction block that replaces everything before it. The context windows page calls it the primary strategy for long-running conversations and agentic workflows.You must pass the compaction block back; a custom instructions string replaces the default summary prompt entirely; compaction is an extra sampling step, so sum usage.iterations for true cost; check the model support list (Haiku 4.5 is not on it as of this writing).
External notes and sub-agentsPersist state outside the window (memory tool, files) or delegate work to sub-agents that return condensed summaries.Needs a retrieval or hand-off design; see 2.4.

Some models also have context awareness: Sonnet 5, Sonnet 4.6, Sonnet 4.5 and Haiku 4.5 track their remaining budget through API-injected tags, which can make them wrap up early near the limit unless your prompt tells them compaction or external state is in play.

Key concept: reduce by information loss, cheapest first

A workable ordering is to prevent tokens entering (trim at the source, load just in time), then remove what is provably dispensable (clear stale tool results), then summarise (compaction), and finally move state outside the window. Each step is lossier than the one before, so apply the least lossy step that solves the problem, and protect critical facts explicitly at every step.

The summarisation trap and long-context failure modes

Summaries compress, and compression drops precisely the details a transactional system needs: amounts, dates, identifiers, promised deadlines. “Customer wants a refund for a recent order” has lost the amount, the order number and the date. The fix is structural. Keep a persistent facts block (a small structured record of the exact values) that you maintain and re-inject on every turn outside the history that gets summarised, and when you use compaction supply custom instructions that name what must be preserved verbatim. Then test it: after a compaction, ask for the exact values and check them.

Other failure modes: stale or duplicated tool output accumulating turn after turn; early errors that persist and get baked into summaries; critical instructions buried mid-context; and long-horizon agents that lose state at a window boundary because nothing was written outside it. Big windows delay these problems, and add cost and latency on every call.

Common exam distractor

“Use a larger context window so nothing needs to be trimmed or summarised” and “tell the model to preserve numbers when summarising” are tempting but wrong. A bigger window postpones the limit while context rot, cost and latency still grow, and an instruction to a summariser is probabilistic. The robust answer is a persistent facts block, targeted trimming or clearing, and a test that verifies the critical values survive.

Exam traps

Practice question

A multi-issue support agent runs long sessions and uses server-side compaction. After a compaction the agent says 'your recent refund request' instead of the $247.83 refund for order #8891. What is the most effective fix?

  • A Switch to a model with a larger context window and raise the compaction trigger so that compaction rarely happens during a typical session, keeping all the detail in context.

    This only postpones the problem. Long sessions will still reach the trigger, and the same details will be lost when they do.

  • B Add a line to the system prompt telling the model never to summarise numbers or identifiers, and to always repeat them exactly and verbatim whenever it is asked.

    The summary is produced by a separate step governed by its own prompt, and instruction wording is probabilistic. It does not protect the values structurally or verify that they survive.

  • C Keep exact facts in a structured block re-sent every turn outside the compacted history, add custom compaction instructions naming them, and test the values after compaction. Correct

    The facts are protected structurally rather than trusted to a summariser, the summary prompt is steered toward what matters (custom instructions replace the default prompt, so they must be complete), and a test verifies the result.

  • D Lower the compaction trigger so summaries happen earlier, when there is less content to compress and each summary has fewer details to lose across the whole session.

    Compacting earlier makes compaction more frequent and does not change what a summary tends to drop. The exact values still need explicit protection.

Build exercise: Instrument and shrink the context of a tool-heavy agent

Intermediate · 75 minutes

You'll practice:

  1. Instrument a 12-turn simulated conversation that calls an order-lookup tool returning 40 fields. Log per-turn usage and use the token counting endpoint to break the request into system prompt, tool definitions and tool results.

    You cannot budget what you have not measured. A breakdown shows which category (fixed, growing, per-request) is actually consuming the window.

    You should see: A table of input tokens per turn that grows steadily, with the tool results as the dominant growing component.

    Hints
    1. Which components of the request stay constant each turn, and which get bigger?
    2. Count tokens for the system prompt alone, for the tools alone, and for the full request; the difference is the conversation and tool results. Count against the model you will actually use.
    3. c = client.messages.count_tokens(model='claude-sonnet-5', system=SYSTEM, tools=TOOLS, messages=messages)
      print(turn, c.input_tokens)
  2. Implement a tool result trimmer that keeps only the five fields the task needs, apply it inside the tool implementation, and re-run the conversation. Compare per-turn input tokens with the baseline.

    Trimming at the source is the least lossy reduction, and it compounds because the trimmed result is not re-sent on every later turn.

    You should see: A visibly flatter growth curve and a percentage reduction you can quote for the whole conversation.

    Hints
    1. Which fields would the agent ever use to answer the customer, and which are internal noise?
    2. Define the whitelist per tool from the task, not from the response shape, and trim before returning the tool result.
    3. KEEP = {'order_id', 'order_date', 'total_amount', 'return_eligible', 'item_description'}
      def trim(result):
          return {k: v for k, v in result.items() if k in KEEP}
  3. Build a survival test for critical facts: replace the first 8 turns with a model-written summary, then ask for the refund amount, order number and date. Run it with and without a persistent facts block re-injected outside the summary.

    This demonstrates the summarisation trap empirically and shows the facts block working, without needing any beta feature.

    You should see: Without the block the agent answers vaguely or wrongly; with the block it states $247.83, #8891 and the correct date.

    Hints
    1. Where does the facts block have to live so it is never part of what gets summarised?
    2. Maintain the block as structured data updated from tool results, and place it in the system prompt or a fixed slot of each request; summarise only the free-text history.
    3. FACTS = {'customer': 'C-4421', 'issues': [{'order': '#8891', 'date': '2025-03-03', 'refund': '$247.83', 'status': 'pending_refund'}]}
      system = BASE + '\n<case_facts>\n' + json.dumps(FACTS) + '\n</case_facts>'
  4. Enable server-side compaction with a low trigger and custom instructions that name what to preserve verbatim. Append each response's content back into your messages, and sum usage.iterations to compute the true cost of a compacting conversation.

    Compaction must be handled correctly (pass the block back) and costed correctly (an extra sampling step is not in the top-level totals). Custom instructions replace the default summary prompt entirely.

    You should see: A compaction block in a response once input passes the trigger, later requests that no longer carry the older turns, and a cost figure larger than the top-level usage alone.

    Hints
    1. What happens to the blocks before the compaction block on the next request, and what must you do with the block itself?
    2. Use the beta messages API with the compaction beta header and the compact edit type; keep the trigger at or above the documented minimum; append response.content as the assistant turn.
    3. resp = client.beta.messages.create(betas=['compact-2026-01-12'], model='claude-sonnet-5', max_tokens=4096, system=SYSTEM, messages=messages,
          context_management={'edits': [{'type': 'compact_20260112', 'trigger': {'type': 'input_tokens', 'value': 50000},
              'instructions': 'Summarise for a support agent. Preserve verbatim: order numbers, amounts, dates, promised deadlines, open issues.'}]})
      messages.append({'role': 'assistant', 'content': resp.content})
      for it in (getattr(resp.usage, 'iterations', None) or []):
          print(it.type, it.input_tokens, it.output_tokens)
  5. Run a placement A/B on 20 questions over a 3-document input: documents before the question versus question before documents, plus a variant that asks for supporting quotes first. Report accuracy and tokens for each.

    Position guidance is directional. Testing it on your own data tells you whether restructuring is worth doing and whether quote-first grounding helps.

    You should see: A small results table showing whether ordering or quote-first changes accuracy on your data, with token counts for each variant.

    Hints
    1. What must stay identical between variants so only ordering differs?
    2. Use the same documents and questions, wrap each document in tags with source metadata, and change only the position of the question and the quote-first instruction.
    3. docs = ''.join(f'<document index="{i}"><source>{s}</source><document_content>{t}</document_content></document>' for i, (s, t) in enumerate(sources, 1))
      prompt_a = '<documents>' + docs + '</documents>\n' + question
      prompt_b = question + '\n<documents>' + docs + '</documents>'

Sources