Study guides / CCDV-F / Domain 2

Model Selection & Optimisation · Lesson 4 of 5

2.4 - Effort and Thinking Budget as Optimisation Levers

Tune thoroughness within a single model using output_config.effort, adaptive thinking, and task budgets - three distinct levers, not one.

Model tier isn't the only optimisation lever — within a single model, the effort parameter, adaptive extended thinking, and (on some models) a task budget all let you tune how much work the model puts into a response, trading cost and latency against thoroughness on a per-request basis rather than a per-deployment one. A high-autonomy coding task genuinely benefits from more effort; a short, well-scoped extraction task doesn't need it and pays only the cost. On current models (Fable 5, Opus 5, Sonnet 5), the older fixed budget_tokens approach to controlling thinking has been removed — effort and adaptive thinking are the current mechanism.

Three different knobs, not one

Effort (output_config: {effort: "low"|"medium"|"high"|"xhigh"|"max"}) scales thinking depth and tool-call behaviour together — lower effort means fewer, more-consolidated tool calls and terser output, not just shorter reasoning. Adaptive thinking (thinking: {type: "adaptive"}) controls whether the model reasons step-by-step before answering at all; on current models it's the only supported "on" mode, replacing the older fixed-budget approach. A task budget (beta, on select models) is a separate, higher-level control: it gives the model a token ceiling for an entire agentic loop so it paces itself across many turns and finishes gracefully, rather than being cut off mid-task. These three are independent — you can raise effort while keeping a tight task budget, or vice versa.

Sweeping effort correctly

Effort should be swept against an evaluation set, one setting at a time, keeping every other request field byte-identical — including the prompt itself, since changing effort mid-conversation invalidates the prompt cache for that conversation going forward. How much effort actually buys varies sharply by workload shape: on research and knowledge-style tasks the accuracy curve tends to be nearly flat, so a lower effort level often captures most of the accuracy at a fraction of the cost. On long-horizon coding and agentic work, effort is a genuine trade-off — dropping from the default to a lower setting can give up real accuracy in exchange for real savings. On tasks that sit right at a model's reasoning ceiling, every additional step of effort can keep buying measurable quality, with no obviously "free" cut available. The exam expects you to know that this curve has to be measured per workload, not assumed.

Exam trap: disabling thinking instead of lowering effort

On Opus 5, thinking is on by default and can be explicitly disabled only at effort high or below. Doing this to save cost has a real failure mode: with thinking disabled, the model can occasionally write what should be a tool call into its visible text instead of a proper tool_use block — the turn completes successfully, the intended tool call never actually runs, no error is raised, and in an agentic loop that stray text can pollute later turns. Lowering effort instead of disabling thinking achieves a similar cost reduction without this failure mode.

Re-running failures at higher effort

When a workload has a usable pass/fail signal — tests, a validator, a checker — one of the most effective effort strategies isn't picking a single setting at all: run everything at a low effort level first, and only re-run the cases that failed at a higher setting. Because most requests succeed on the first, cheap attempt, and only the harder tail needs the expensive retry, this pattern captures most of the accuracy of always running at the higher setting while paying the higher cost only where it's actually needed. It trades the saving for extra wall-clock time on the failed cases (since those pay for two attempts, not one) and it depends entirely on having a cheap, reliable way to detect failure — without that signal, this strategy can't tell a real failure from a plausible-looking wrong answer, and shouldn't be used.

Task budgets vs. max_tokens

These are easy to conflate but solve different problems. max_tokens is an enforced per-response ceiling — the model has no awareness of it, and hitting it truncates output abruptly. A task budget is advisory and token-denominated: the server injects a countdown the model can see during generation, so it paces its own work across a whole agentic loop and tries to land within the ceiling rather than being cut off. Task budgets have a floor (a minimum total of 20,000 tokens) and are set once at the start of a task — changing the budget mid-task, like changing effort mid-conversation, invalidates the cache. A generous budget gives up little accuracy for a modest saving; a very tight budget can meaningfully hurt pass rate and, in the extreme, produce refusal-like behaviour as the model runs out of room to work.

Key concept

Think of tier as "which model," effort as "how hard that model tries on this request," and task budget as "how much room it's given to try across an entire loop" — three separable decisions, not one.

Exam traps

Practice question

An autonomous coding agent occasionally produces shallow, incomplete refactors on genuinely complex multi-file changes, but works well on smaller changes. The team wants to fix this without changing which model is configured. What's the appropriate lever?

  • A Increase the effort/thinking budget for this task, so the same model spends more work on complex cases. Correct

    This tunes thoroughness within the already-chosen model, which directly targets the described problem - shallow output specifically on the harder cases - without a tier change.

  • B Increase max_tokens only.

    A higher output cap allows a longer response but doesn't itself make the model reason more thoroughly about a complex change.

  • C Disable prompt caching so the model reprocesses the full context every time.

    Caching affects cost/latency of re-processing input, not reasoning thoroughness - this wouldn't address shallow output.

  • D Switch to streaming responses.

    Streaming changes delivery, not reasoning depth - it wouldn't fix shallow refactors.

Build exercise: Compare effort levels, task budgets, and the thinking-disabled pitfall on the same model

Intermediate · 35 minutes

You'll practice:

  1. Pick a task with real complexity (a multi-step planning or refactor description) and run it on Claude Opus 5 at effort 'low' and then effort 'high', keeping the prompt identical.

    This isolates the effect of the effort lever from the model-tier lever covered in Lesson 2.1, and gives you a concrete baseline before exploring the other levers in this exercise.

    You should see: A visible difference in thoroughness, output_tokens, and latency between the two runs.

    Hints
    1. Where in the request does the effort setting live, and what should you keep fixed between the two calls?
    2. effort lives inside output_config. Keep the model and the prompt identical between calls so only effort varies.
    3. complex_task = "Refactor this module description into a migration plan across 4 services with rollback steps: ..."
      
      for effort in ["low", "high"]:
          r = client.messages.create(
              model="claude-opus-5",
              max_tokens=4000,
              output_config={"effort": effort},
              messages=[{"role": "user", "content": complex_task}],
          )
          text = next(b.text for b in r.content if b.type == "text")
          print(effort, "output_tokens:", r.usage.output_tokens, "len:", len(text))
  2. Give the model a tool and send a tool-triggering prompt with thinking explicitly disabled at a mid effort level, then inspect response.content block types to check whether the tool call arrived as a proper tool_use block or leaked into text.

    This makes the exam-relevant pitfall concrete rather than abstract - you should be able to recognise this failure mode if it shows up in a scenario question.

    You should see: In most runs a clean tool_use block, but occasionally text content that describes or contains what should have been a tool call - worth running a few times to see the variability.

    Hints
    1. What are the possible values of block.type in response.content, and which one indicates a real tool call happened?
    2. Iterate response.content and print each block's type. A tool_use block means the call happened correctly; a text block containing tool-call-shaped content is the failure mode from the lesson.
    3. r = client.messages.create(
          model="claude-opus-5",
          max_tokens=2000,
          thinking={"type": "disabled"},
          output_config={"effort": "medium"},
          tools=[my_tool],
          messages=[{"role": "user", "content": "Look up the account balance for customer 4021."}],
      )
      for block in r.content:
          print(block.type, getattr(block, "text", None) or getattr(block, "name", None))
  3. Set up a streaming request with a task_budget of 40,000 tokens using the beta header, and read the final usage after the stream completes.

    Task budgets require streaming (to avoid HTTP timeouts on large max_tokens) and demonstrate the advisory, self-pacing behaviour that distinguishes a budget from a hard max_tokens cutoff.

    You should see: A completed response with usage.output_tokens landing at or under the budget in most runs, without a truncated stop_reason.

    Hints
    1. Which client method do you need for a beta feature that requires streaming, and what beta flag does a task budget need?
    2. Use client.beta.messages.stream with the task-budgets beta flag, nest task_budget inside output_config alongside effort, and call stream.get_final_message() once the stream ends.
    3. with client.beta.messages.stream(
          model="claude-opus-5",
          max_tokens=64000,
          output_config={"effort": "high", "task_budget": {"type": "tokens", "total": 40000}},
          betas=["task-budgets-2026-03-13"],
          messages=[{"role": "user", "content": "Plan and execute a multi-file refactor..."}],
          tools=[my_tool],
      ) as stream:
          final = stream.get_final_message()
      print(final.usage.output_tokens)
  4. Run a sweep across effort 'low', 'medium', and 'high' on the same complex task from step 1, recording output_tokens for each, without changing anything else.

    A three-point sweep is the minimum needed to see whether a workload's accuracy/cost curve is flat, steep, or somewhere in between - the shape the lesson says you must measure rather than assume.

    You should see: A dictionary of three effort levels each mapped to a token count, likely increasing from low to high.

    Hints
    1. How would you structure a loop so you can compare all three settings side by side afterward?
    2. Loop over the three effort strings, store each result's output_tokens in a dict keyed by effort, then print the whole dict at the end.
    3. results = {}
      for effort in ["low", "medium", "high"]:
          r = client.messages.create(
              model="claude-opus-5", max_tokens=4000,
              output_config={"effort": effort},
              messages=[{"role": "user", "content": complex_task}],
          )
          results[effort] = r.usage.output_tokens
      print(results)
  5. Start a cached conversation at effort 'high', continue it at effort 'low' on the next turn, and check whether cache_read_input_tokens on the second call is zero.

    This proves the exam-relevant claim that changing effort mid-conversation invalidates the prompt cache - a cost side-effect of the effort lever that's easy to miss.

    You should see: cache_read_input_tokens at or near zero on the second call, even though the cached system prompt content didn't change - the effort change alone broke the cache.

    Hints
    1. What request field, besides the message content itself, can also invalidate a cache hit between two calls in the same conversation?
    2. Send the first call at effort 'high' with a cached system block, append its response to messages, then send a follow-up at effort 'low' with the same cached system block and compare cache_read_input_tokens.
    3. messages = [{"role": "user", "content": "Long task with a big cached system prompt..."}]
      
      r1 = client.messages.create(model="claude-opus-5", max_tokens=1024, system=[{"type": "text", "text": BIG_SYSTEM_PROMPT, "cache_control": {"type": "ephemeral"}}], output_config={"effort": "high"}, messages=messages)
      messages.append({"role": "assistant", "content": r1.content})
      messages.append({"role": "user", "content": "Continue, but now at lower effort."})
      
      r2 = client.messages.create(model="claude-opus-5", max_tokens=1024, system=[{"type": "text", "text": BIG_SYSTEM_PROMPT, "cache_control": {"type": "ephemeral"}}], output_config={"effort": "low"}, messages=messages)
      print("r2 cache_read_input_tokens:", r2.usage.cache_read_input_tokens)

Sources