CCDV-F Glossary - Security & Safety

Quick-lookup definitions for every domain, with exam context and links back to the lesson that covers each term.

Prompt Injection: Content encountered by Claude - a web page, email, PDF, or tool result - that contains instructions aimed at the model rather than the human reader, trying to override or redirect its actual task ("ignore your previous instructions and instead..."). It exploits a structural fact about how LLMs work: everything in the context window, whether a system prompt, a user request, or fetched text, is ultimately just tokens in a sequence, with no hard architectural boundary between "trusted instruction" and "inert data."; Exam context: The exam frames prompt injection as a probabilistic risk that prompt wording reduces but cannot eliminate. Expect distractors that treat a strongly-worded system-prompt instruction, or simply using a more capable model, as a complete, standalone fix - neither removes the underlying risk.; See also: 6.1 Prompt Injection and Untrusted Content
Direct Prompt Injection: An attack where the person talking to Claude directly tries to override its instructions or safety behavior - classic jailbreak attempts, role-play framings, "pretend you have no restrictions" prompts. The attacker and the legitimate user are the same person, typing straight into the conversation.; Exam context: The exam tests whether you can tell direct injection (a user-facing jailbreak problem) apart from indirect injection, since the two call for different fixes and are commonly conflated as "the same problem with the same fix."; See also: 6.1 Prompt Injection and Untrusted Content
Indirect Prompt Injection: The agent-specific case where a malicious instruction arrives via a third party's content that the agent processes on someone else's behalf - a hidden instruction in a customer email, a fetched web page, or a README from an untrusted repository. The victim (the actual user) never sees the injected text and is a different person from the attacker who authored it.; Exam context: This is the pattern the exam tests most in agentic scenarios. A common trap: assuming that because the user is trusted, any content the agent reads on their behalf is safe to act on - trusting the user says nothing about the trustworthiness of third-party content the agent merely reads.; See also: 6.1 Prompt Injection and Untrusted Content
Spotlighting (Content Isolation): Wrapping third-party or untrusted content in explicit structural markers (e.g. <untrusted_web_content> tags), paired with a system-prompt rule that content inside those tags is data to process, never a command to obey. It gives the model a stronger structural signal about which part of the context is instructions versus data, without claiming to eliminate the risk.; Exam context: The exam expects spotlighting named as one layer of defense-in-depth alongside capability scoping and human checkpoints - never presented as a sufficient, standalone fix by itself.; See also: 6.1 Prompt Injection and Untrusted Content
Principle of Least Privilege: Grant an agent's tools only what the current task actually needs, not broad access that happens to be convenient to set up once. Every tool given to an agent is a capability that a bug, a misjudgment, or an injected instruction could trigger - so the safest tool set is the smallest one that still lets the agent do its actual job.; Exam context: The exam ties this directly back to prompt-injection defense: an agent that structurally lacks a dangerous tool cannot be tricked into using it, no matter how convincing an injected instruction is. This is treated as the real backbone of injection defense, not a separate topic.; See also: 6.2 Tool Permissions: Least Privilege, Allowlists and Denylists
Allowlist: A permission model where only specific, explicitly-listed actions are permitted and everything else is blocked by default - it "fails closed." An action nobody thought to list is denied, not allowed.; Exam context: The exam's safe default for any high-consequence tool set is an allowlist. Watch for distractors favoring a denylist because it's "shorter" or "easier to maintain" - ease of authoring doesn't offset the structural risk of failing open.; See also: 6.2 Tool Permissions: Least Privilege, Allowlists and Denylists
Denylist: A permission model where everything is permitted except specific, explicitly-listed blocked actions - it "fails open." Anything not anticipated at design time, including a newly added tool or an unexpected combination of two individually-safe actions, is allowed by default.; Exam context: A common wrong answer treats a denylist as sufficient for a consequential tool set because it's quicker to write. The exam tests recognizing that a denylist requires having already imagined every bad outcome in advance - a much harder bar than it sounds.; See also: 6.2 Tool Permissions: Least Privilege, Allowlists and Denylists
Blast Radius (Parameter- and Credential-Level Scoping): The worst-case outcome if a given tool call executes with attacker-chosen parameters. Least privilege doesn't stop at whether an agent has a tool at all - a tool can be over-scoped through unrestricted parameters (a general query_database(sql) vs. a narrow get_order_status(order_id)) or an overly-broad underlying credential (an admin database role instead of a genuinely read-only one).; Exam context: The classic exam distractor treats a tool's description, or a system-prompt instruction like "only use this for reads," as an enforcement boundary. Real enforcement lives in the tool's implementation and credential scope - not in prose a model mistake or an injected instruction can talk its way around.; See also: 6.2 Tool Permissions: Least Privilege, Allowlists and Denylists
Guardrail: A mechanism that sits outside the model's own probabilistic judgment and enforces a policy deterministically - in code, not in the prompt - such as a hook that blocks a tool call, logs a consequential action, or redacts sensitive data from a tool result before it re-enters the model's context.; Exam context: The exam frames the choice between a prompt instruction and a guardrail around one question: would a single failure cause real financial, legal, or safety harm the business cannot absorb? If yes, that requirement needs a hook, not just careful wording, because prompt instructions are probabilistic and can't guarantee compliance.; See also: 6.3 Guardrails: Hooks and Human-in-the-Loop for Risky Actions
PreToolUse Hook: An interception point that runs before a tool executes, able to block, modify, or redirect the outgoing call - the underlying tool simply never runs if the hook decides to stop it. It's the only hook direction that can prevent a consequential action, because by definition nothing has happened yet.; Exam context: If a requirement is "this must never happen," the exam expects a PreToolUse gate as the answer. A PostToolUse hook offered as the fix for prevention is a built-in wrong answer - it's structurally too late.; See also: 6.3 Guardrails: Hooks and Human-in-the-Loop for Risky Actions
PostToolUse Hook: An interception point that runs after a tool executes but before the model processes the result - the right tool for normalizing data formats, redacting sensitive fields, or logging what happened for audit. It cannot undo an action that has already taken place, even if it detects a policy violation.; Exam context: The exam's most direct trap on this domain: presenting a PostToolUse hook as a way to prevent a policy-violating action from executing. It's wrong by construction - the non-compliant action has already occurred by the time the hook fires.; See also: 6.3 Guardrails: Hooks and Human-in-the-Loop for Risky Actions
Human-in-the-Loop (Consequence and Reversibility): A human approval checkpoint reserved for actions that are both high-consequence (real financial, legal, safety, or reputational harm) and hard-to-reverse (a sent email, an executed transfer, a deleted production record) - not applied uniformly to every tool out of general caution. Two patterns implement it: synchronous (blocking) approval for single high-stakes actions, and tiered thresholds that route different risk levels to different sign-off requirements (e.g. auto-approve under $50, manager sign-off above $500).; Exam context: Expect scenarios asking where to place a gate among several tools of differing risk (read-only, draft, send) - the correct answer targets only the irreversible, externally-visible action. Gating everything regardless of consequence causes approval fatigue, where reviewers rubber-stamp without real scrutiny, which the exam treats as a genuine failure mode, not a safe-by-default choice.; See also: 6.3 Guardrails: Hooks and Human-in-the-Loop for Risky Actions