PPassTheCertExam

Study guides / CCAR-P / Domain 1

Integration · Lesson 1 of 8

1.1 - Auditing Tool and Agent Configuration for Capability Bloat

Audit an agent's tool and MCP configuration for capability bloat, diagnose whether the problem is token cost, selection accuracy or attack surface, and choose an enforceable remedy.

Capability bloat is what accumulates when an agent is given more than the task needs: dozens of tool definitions loaded on every request, one agent that owns every tool in the system, MCP servers connected "just in case". It rarely fails loudly. It shows up as three separate costs, and a scenario question will try to blur them, so learn to name each one.

The architect's job is to find which cost is actually hurting, because the remedy differs. Credentials are Lesson 1.2; load-on-demand mechanics are Lesson 1.8.

Diagnose before you prune

The same visible symptom ("the agent picks the wrong tool") has different root causes. Match the cause to the remedy before touching anything.

What you observeRoot causeRemedy
Two tools with similar names get confused; the toolkit is smallAmbiguous interfaceRewrite descriptions (what it does, when to use it and when not to, what each parameter means; Anthropic recommends at least 3–4 sentences per tool), and rename or namespace
Many tools that are variations of one job (pivot, percentile, normalise…)Decision space wider than the jobConsolidate into fewer tools with an action or enum parameter
Ten or more tools, or tens of thousands of definition tokens, most unused on any one requestEverything loaded up frontTool search with defer_loading; keep the 3–5 most-used tools loaded
One agent holds tools for several unrelated rolesOverbroad agentSplit by role into subagents, each with its own tool allowlist
A tool can do far more than the job (arbitrary SQL, fetch any URL, write anywhere)Over-scoped toolReplace with a constrained tool, and enforce the limit at the credential or configuration layer

Watch the tension in consolidation. It shrinks what the model chooses between, but it merges capabilities: once manage_orders(action=...) covers read, refund and delete, a permission rule that matches on tool name can no longer allow one and deny another. Consolidate within a risk class and keep destructive operations as their own tool.

What does not reduce bloat: moving tools to a second MCP server. The host merges all servers' tools into one registry, so 22 tools over two servers is still 22 tools.

How many tools is too many?

Anthropic documents no fixed per-agent tool count, so treat any single number you have seen quoted (such as 4–5 tools) as a design heuristic, not a limit. What the docs do give are softer figures: degraded selection beyond roughly 30–50 tools, and tool search recommended from 10 or more tools or more than about 10k tokens of definitions; standard loading fits fewer than 10 tools or tools used on every request.

All of them agree on the pattern: a small, role-scoped set in front of the model and the long tail discoverable rather than resident. A scenario will hinge on that pattern, not the digit; use any numbers in the stem to pick which remedy row applies.

Key concept: three quantities, not one

Tool count is only a proxy. What you are actually managing is (1) decision ambiguity (can a human say which tool applies?), (2) tokens carried per request, and (3) blast radius if a call is wrong or hijacked. Anthropic's own test for the first: "If a human engineer can't definitively say which tool should be used in a given situation, an AI agent can't be expected to do better."

A repeatable audit method

  1. Inventory what the model sees. One flat list per agent, including every connected MCP server's tools, with owner and whether each reads, writes or destroys.
  2. Measure token cost. The token counting endpoint accepts a tools array, so count with and without a tool set. It is an estimate and rejects server tools and the MCP connector; for those read usage on real responses.
  3. Measure use. Log which tool each agent calls per task type. Never-called tools are removal candidates; always-together tools are consolidation candidates.
  4. Measure ambiguity. Run an eval set with the expected tool per request. Confusion between two tools is a description problem; scattered errors across a large catalogue is a scale problem.
  5. Classify privilege. What would the worst call with attacker-chosen parameters do, and which credential sits behind it?
  6. Apply the smallest change that fixes the measured problem, by blast radius: disable, constrain, split by role, defer, then rewrite.
  7. Re-run the evals for accuracy, cost and latency, plus a security probe: does an injected request for a removed tool have any tool to call?

Decide the tool set per agent or session, not per turn: changing tool definitions mid-conversation invalidates the prompt cache, and the miss can cost more than the definitions you dropped (the MCP client guidance says so). Deferred loading is the mechanism built to avoid that.

Enforcement, not instruction

Pruning counts only if it is enforced where the model cannot argue. "Only use this for reads" in a system prompt is guidance, not a control. The real levers are configuration:

Attack surface matters most where tool output is untrusted. A tool that reads tickets, web pages or third-party MCP results is a channel for injected instructions, so the fewer write-capable tools behind that channel the better. Fewer, narrower tools is the cheapest guardrail you have (the full layers are Domain 4).

Common exam distractor

Watch for answers that treat bloat as a wording problem when it is a scale or privilege problem: "rewrite all 90 descriptions", "move half the tools to a second server", "tell the agent not to use the write tools". The inverse trap is one mega-tool with an action parameter, which trades permission granularity for fewer tools. The right answer names the cause first, then applies a change the model cannot bypass.

Exam traps

Practice question

A platform team's assistant is connected to six MCP servers exposing about 120 tools. Definitions consume tens of thousands of input tokens on every request, the assistant frequently picks a plausible but wrong tool, and a security review flags that a third-party ticketing server exposes tools that can delete and reassign tickets, which the assistant never needs. Logs show that five tools account for most calls. Which plan best addresses all three concerns?

  • A Rewrite all 120 tool descriptions to be more detailed and consistent, add a system prompt line telling the assistant never to delete or reassign tickets, and re-test on a sample of ticket workflows

    Longer descriptions add tokens to a request that is already too large, and they do not shrink the decision space. The system prompt line is guidance, not enforcement, so the delete and reassign tools remain callable.

  • B Keep the five most-used tools loaded and defer the long tail behind tool search; disable the delete and reassign tools in the toolset configuration; then re-run a selection eval and a token count Correct

    It matches each remedy to the measured cause: deferred loading addresses token cost and selection at scale, the toolset configuration removes the unneeded privileged capability in a way the model cannot override, and the re-run verifies the change instead of assuming it.

  • C Move the third-party ticketing tools to a separate MCP server so the model treats them as a distinct group, and document which server owns which tools

    The model sees one flat list regardless of which server a tool comes from. Nothing about tokens, accuracy or privilege changes.

  • D Merge the 120 tools into a single tool with an action parameter so the model only ever chooses between that one tool and a plain reply

    Consolidation is useful for near-duplicate operations, but collapsing everything hides the real decision inside a huge enum, and it merges destructive and read-only operations so they can no longer be permitted separately.

Build exercise: Audit and prune a 40-tool agent configuration

Intermediate · 75 minutes

You'll practice:

  1. Take a real or invented agent with at least 30 tools across two or more MCP servers and write the inventory: tool name, server, owning agent, and a risk class (read, write, destructive, external network).

    You cannot prune what you have not listed, and the model sees one flat list regardless of server boundaries. The risk class is what later drives disable-versus-defer decisions.

    You should see: A table with one row per tool, sorted so that near-duplicates and destructive tools are easy to spot, and a note on which tools are third-party.

    Hints 
    1. Which columns would let you later answer 'who could be harmed if this tool is called wrongly?' and 'which tools does this agent even need?'
    2. Include: name, server, agent(s) that can call it, read/write/destructive, and whether its output is untrusted content (web, tickets, email).
    3. Example row: get_order_status | orders-mcp | support-agent | read | output trusted. Example row: ticket_delete | ticketing-3p | support-agent | destructive | third-party server. Flag every destructive row for review.

  2. Measure the token cost of the tool definitions with the token counting endpoint: count one request with the full tools array and one with a single-tool baseline, and compute the difference.

    Cost is the easiest of the three bloat costs to quantify, and a number makes the trade-off discussable. The count is an estimate, not a bill.

    You should see: A single figure for the input tokens attributable to the tool set, plus the same figure per server if you count each server's tools separately.

    Hints 
    1. How can you isolate the effect of the tools from the effect of the message and system prompt?
    2. Send the same message and system prompt twice through client.messages.count_tokens, once with all tools and once with only one tool, and subtract the two input_tokens values. Repeat per server to see which server dominates.
    3. import anthropic
client = anthropic.Anthropic()
def count(tools):
    r = client.messages.count_tokens(model='claude-sonnet-5', tools=tools, messages=[{'role':'user','content':'ping'}])
    return r.input_tokens
full, base = count(all_tools), count(all_tools[:1])
print('definition tokens ~', full - base)
Remember the endpoint rejects server tools and the MCP connector; for those, read usage.input_tokens from real responses.

  3. Gather usage evidence: run 30 or more representative tasks (or replay logs) and record which tool each agent calls. Mark tools with zero calls, and pairs of tools that are frequently confused.

    Usage and confusion data separate dead weight from real capability, and separate a description problem (two tools confused) from a scale problem (errors scattered across a large catalogue).

    You should see: A per-tool call count, a list of zero-use tools, and a short confusion list naming the tool the agent chose versus the tool a human would have chosen.

    Hints 
    1. What would distinguish 'tool is unused because it is not needed' from 'tool is unused because the model cannot tell it apart from another'?
    2. For each zero-use tool, check whether a task in your sample should have used it. If yes, suspect ambiguity; if no, suspect dead weight.
    3. Keep a two-column log: expected_tool, chosen_tool. Compute how many misses land on the same two tools (description problem) versus how many are spread over many tools (scale problem).

  4. Write the remedy for every tool group using the diagnosis table, then produce the target configuration: which 3 to 5 tools stay loaded, which are deferred, which subagent gets which allowlist, and which tools are disabled outright.

    This is the decision the exam asks you to make and justify. Each change must map to a measured cause, and enforcement must sit in configuration.

    You should see: A one-page target configuration, with an MCP toolset allowlist or denylist snippet and a subagent tools list, and a one-line justification per change.

    Hints 
    1. Which of your findings is about ambiguity, which about scale, and which about privilege?
    2. Ambiguity: rewrite or rename. Scale: defer_loading behind tool search. Privilege: disable in the toolset (default_config.enabled false plus explicit enables) or remove from the subagent's tools list.
    3. {"type":"mcp_toolset","mcp_server_name":"ticketing-3p","default_config":{"enabled":false,"defer_loading":true},"configs":{"ticket_search":{"enabled":true,"defer_loading":false},"ticket_get":{"enabled":true}}}
Allowlist: everything is off unless named, so a tool the server adds later stays off.

  5. Design the regression check: the eval that proves selection accuracy did not fall, the token count that proves cost fell, and a security probe that shows the removed capability is unreachable.

    Pruning is a change to a probabilistic system. Without a before-and-after measurement you have only a hypothesis, and without a probe you have only an instruction.

    You should see: A short test plan with three checks (selection accuracy, tokens and latency, security probe), each with a pass condition.

    Hints 
    1. What single result would make you roll the change back?
    2. Reuse the expected_tool set from step 3 for selection accuracy. Recount tokens with the new tools array. For the probe, send a request that tries to make the agent delete a ticket and verify that no delete tool is in its tool list.
    3. Pass conditions: selection accuracy on the eval set is not lower than the baseline, definition tokens fall by the amount predicted in step 2, and the injected delete request produces no tool_use for a destructive tool because none exists in the session.

Sources