Study guides / CCDV-F / Domain 4

Prompt & Context Engineering · Lesson 4 of 4

4.4 - Context Window Management

Recognise the progressive-summarisation trap and the lost-in-the-middle effect before they quietly degrade a long-running agent.

A long conversation or agent session accumulates context that eventually has to be managed, not just resent forever — both because the context window has a hard token ceiling and because a very long, undifferentiated context degrades output quality well before it hits that ceiling. Two specific failure modes matter for the exam: the progressive summarisation trap, where repeatedly summarising an already-summarised history compounds information loss until important details are quietly gone; and the lost-in-the-middle effect, where content placed in the middle of a very long context gets weighted less reliably by the model than content near the start or end.

Why progressive summarisation compounds loss

Summarising a conversation once is a reasonable compression step — you trade some detail for a much smaller token footprint, and for most agent tasks the tradeoff is worth it. The trap is doing it repeatedly, using each summary as the input to the next: summary-of-summary-of-summary. Each pass only has access to what the previous summary chose to keep, not the original source material, so anything that got compressed away on pass one can never come back on pass two — there's no mechanism to recover it. Worse, there's no visible failure signal when this happens. The agent doesn't error; it just quietly starts reasoning from a thinner and thinner version of its own history, and the first sign of trouble is often the agent contradicting or forgetting something it correctly established many turns earlier.

Lost-in-the-middle: why position matters

Within a single very long context, content near the beginning and content near the end tend to be attended to more reliably than content buried in the middle — a well-documented pattern across long-context language models generally, not a quirk unique to any one system. The practical implication for prompt and context design: don't assume a fact stated once, early in a long conversation, will carry equal weight forty turns later just because it's technically still present in the context. If something genuinely matters late in a long session — a hard constraint, a corrected fact, a user preference — consider re-stating it near the end of the context (e.g. folding it into a system-reminder-style message just before the final turn) rather than trusting it to still carry full weight from wherever it first appeared.

Trimming stale tool results: the highest-leverage technique

In an agentic loop specifically, tool results — not conversational text — are usually the biggest driver of context growth. A large file read, a verbose API response, or a long search result gets appended to history once and then resent, unchanged, on every subsequent API call, even long after the agent has extracted what it needed and moved on. Deliberately trimming or summarising specifically stale tool results — replacing a large, no-longer-needed result with a short placeholder — is usually the single highest-leverage technique for keeping a long agent session's context under control, and it's more surgical than summarising the whole conversation (which risks the progressive-summarisation trap above if applied repeatedly to the same content).

// After the agent has moved past needing this tool result's full content:
messages[toolResultIndex] = {
  ...messages[toolResultIndex],
  content: [{
    type: "tool_result",
    tool_use_id: originalToolUseId,
    content: "[full file contents omitted - previously read in full, 4,200 tokens]",
  }],
};

Keep enough of a placeholder that the model still knows the step happened and roughly what it returned — deleting the turn outright can confuse a model reasoning about what it has and hasn't already done.

The prompt-caching interaction

Prompt caching keys off a stable prefix: everything up to a cache_control breakpoint has to match exactly, byte for byte, for a cache hit. Trimming or rewriting a tool result earlier in the message list changes that prefix, which invalidates the cache for every turn from that edit point onward — the next call re-processes (and re-pays for) everything after the edit as an uncached prefix. This is a real cost tradeoff, not just a mechanical detail: aggressive per-turn trimming can save input tokens on one call while quietly destroying cache savings on the next. The practical pattern is to trim in batches at natural checkpoints (e.g. after a phase of the task completes) rather than on every single turn, and to place your cache breakpoint after the stable system prompt/tool-definitions block and before the volatile, frequently-trimmed tail of the conversation.

Common exam distractor

Trimming the oldest messages indiscriminately (a simple sliding window over turns) is a common but imprecise answer — it can just as easily discard a short, still-relevant early instruction while leaving a huge, already-stale tool result sitting untouched several turns later. The exam favours targeting the actual driver of token growth (large stale tool results) over blindly trimming by recency alone.

Key concept

Manage context deliberately, not reflexively: trim large stale tool results specifically rather than summarising the whole conversation repeatedly, restate genuinely important late-session facts near the end rather than trusting mid-context recall, and remember that editing earlier turns has a prompt-caching cost, not just a token-count benefit.

Exam traps

Practice question

A long-running research agent periodically compresses its own history by re-summarising the running summary plus new findings, to keep context manageable. After many cycles, it starts contradicting an early finding it had correctly identified much earlier. What's the most likely cause?

  • A The model's context window shrank over the course of the session.

    Context window size is fixed per model/request, not something that shrinks during a session - this isn't the mechanism.

  • B Repeated re-summarisation of an already-summarised history compounded information loss until the early finding's detail was dropped. Correct

    This is exactly the progressive summarisation trap - each pass over an already-lossy summary can drop more detail, and there's no built-in signal that something important was lost.

  • C Prompt caching served a stale cached version of the early finding.

    A stale cache would serve old content unchanged, not cause the model to contradict a finding - the described symptom is information loss, not stale reuse.

  • D The agent's tool_choice setting was misconfigured.

    tool_choice controls whether/which tool gets called; it has no bearing on information surviving across repeated summarisation.

Build exercise: Trim stale tool results instead of the conversation text

Intermediate · 40 minutes

You'll practice:

  1. Simulate an agent loop where an early tool call returns a large result (e.g. a long file's contents, a few thousand tokens), followed by several more turns that no longer need that specific content. Log the input_tokens field from the API response's usage object on each call.

    You need a concrete before-number to compare against - the usage.input_tokens field is the ground truth for how much the large tool result is costing you on every subsequent call.

    You should see: input_tokens climbing and staying high across later turns, even though the large tool result from turn 2 is no longer relevant to what the agent is doing by turn 6.

    Hints
    1. Which field on the API response object tells you exactly how many input tokens a given call consumed?
    2. Every messages.create response includes a usage object with input_tokens and output_tokens - log usage.input_tokens after each call in your simulated loop and watch it stay elevated even as the conversation moves on.
    3. for (const turn of simulatedTurns) {
        const res = await client.messages.create({ model: "claude-sonnet-5", max_tokens: 1024, messages });
        console.log(`turn ${turn}: input_tokens=${res.usage.input_tokens}`);
        messages.push({ role: "assistant", content: res.content });
        // ... append next turn ...
      }
  2. Replace the stale large tool result in message history with a short placeholder string before the next call, keeping the tool_use_id intact so the message pairing stays valid.

    This is the highest-leverage context-management technique in the lesson - surgical, targeted trimming of the specific thing driving growth, rather than summarising the whole conversation.

    You should see: A meaningfully smaller input_tokens count on the next call compared to leaving the large result in place, with no loss of the agent's ability to complete the remaining steps.

    Hints
    1. What's the minimum information the model still needs to know that step happened, versus the full content it no longer needs?
    2. Find the tool_result content block in your messages array by its tool_use_id and replace only its content field with a short string like "[full file contents omitted, previously read]" - leave tool_use_id unchanged so the pairing with the earlier tool_use block stays valid.
    3. const idx = messages.findIndex(m => m.role === "user" && m.content.some(b => b.type === "tool_result" && b.tool_use_id === staleId));
      messages[idx].content = messages[idx].content.map(b =>
        b.tool_use_id === staleId ? { ...b, content: "[full file contents omitted, previously read]" } : b
      );
  3. Reproduce the progressive-summarisation trap directly: take a 6-turn conversation, summarise it, then summarise that summary plus 3 new turns, then summarise that result plus 3 more turns. Compare a specific detail from turn 1 across all three summary versions.

    Seeing the detail degrade or vanish across summary generations makes the compounding-loss mechanism concrete rather than abstract, and shows why this differs from the targeted-trimming approach in the previous step.

    You should see: The specific detail from turn 1 (e.g. an exact figure or a named constraint) present in full in summary 1, present but vaguer in summary 2, and missing or altered in summary 3.

    Hints
    1. Pick one very specific, checkable fact to track (an exact number or a named entity), not a vague theme - vague themes are harder to tell apart across summary versions.
    2. Ask the model to summarise the conversation-so-far each time, feeding only the previous summary plus new turns (never the original raw turns) into the next summarisation call, exactly reproducing the compounding structure.
    3. let summary = await summarise(turns.slice(0, 6));
      console.log("gen 1:", summary);
      summary = await summarise([summary, ...turns.slice(6, 9)].join("\n"));
      console.log("gen 2:", summary);
      summary = await summarise([summary, ...turns.slice(9, 12)].join("\n"));
      console.log("gen 3:", summary);
      // check whether the turn-1 detail still appears verbatim in gen 3
  4. Build a long synthetic context (a few dozen short turns) with one critical fact planted near the middle and nowhere else, then ask a question that depends on that fact - first without restating it, then with it restated in a final system-reminder-style message just before the question.

    This directly tests the lost-in-the-middle effect and its practical mitigation - restating a genuinely important fact near the end of a long context rather than trusting mid-context placement.

    You should see: Noticeably less reliable use of the planted fact when it's only present in the middle, versus consistently correct use once it's restated near the end.

    Hints
    1. Make the planted fact something the model can't guess or infer from general knowledge - an arbitrary made-up constraint or number works well for a clean test.
    2. Run the same final question multiple times against the middle-only version and the restated version, and compare how often the answer correctly reflects the planted fact in each case.
    3. const midOnly = [...paddingTurnsBefore, plantedFactTurn, ...paddingTurnsAfter, questionTurn];
      const restated = [...paddingTurnsBefore, plantedFactTurn, ...paddingTurnsAfter, { role: "user", content: `Reminder: ${plantedFact}\n\n${questionText}` }];
      // run both several times and compare how often the fact is correctly used
  5. Add a cache_control breakpoint after the stable system prompt/tool-definitions block, then compare cache read/write token counts (from usage.cache_read_input_tokens and usage.cache_creation_input_tokens) when trimming a tool result every turn versus trimming only once every 5 turns.

    This makes the caching tradeoff concrete: per-turn trimming maximises immediate token savings but invalidates the cache on every call, while batched trimming preserves more cache hits at the cost of carrying stale content slightly longer.

    You should see: Near-zero cache_read_input_tokens (frequent cache misses) under per-turn trimming, versus non-zero cache_read_input_tokens on most turns under batched trimming - a visible cost tradeoff, not just a theoretical one.

    Hints
    1. Which usage fields on the API response distinguish a cache hit from a full re-processing of the prefix?
    2. Set cache_control: { type: "ephemeral" } on the last content block of your system prompt / tool definitions, then run the same simulated session twice - once trimming a stale tool result on every turn, once trimming only every 5th turn - logging usage.cache_read_input_tokens each time.
    3. const system = [{ type: "text", text: systemPromptText, cache_control: { type: "ephemeral" } }];
      for (const turn of turns) {
        const res = await client.messages.create({ model: "claude-sonnet-5", max_tokens: 1024, system, tools, messages });
        console.log(`cache_read=${res.usage.cache_read_input_tokens}, cache_creation=${res.usage.cache_creation_input_tokens}`);
        // trim on every turn in run A, only every 5th turn in run B
      }

Sources