CCAO-F Glossary - Troubleshooting and Optimisation

Quick-lookup definitions for every domain, with exam context and links back to the lesson that covers each term.

Try-Notice-Adjust Loop: A repeatable diagnostic cycle for troubleshooting a prompt: try it against a real input, notice precisely where the output falls short (not just that it's "bad"), and adjust with one targeted change aimed at that specific shortfall, then run the loop again. It replaces ad-hoc tweaking, where changing several things at once because an output "felt off" fixes the one case you were staring at while quietly breaking something else.; Exam context: The exam favours a precise, one-change-at-a-time diagnosis over a full prompt rewrite triggered by a single bad output. Expect distractors describing a ground-up rewrite immediately after noticing one problem, with no named cause.; See also: 7.1 Diagnosing an Underperforming Prompt
Diagnosing Against Real Failing Examples: Asking Claude to help troubleshoot its own underperforming prompt by pasting in the prompt together with one or two concrete outputs you weren't happy with, so Claude can explain the likely cause of each specific problem and suggest a targeted fix. Given real failures to reason from, Claude typically points to a missing role, missing examples, ambiguous instructions, or no explicit criteria as concrete diagnoses.; Exam context: A common trap is asking generically ("here's my prompt, make it better") with no failing example attached — without a concrete instance of the output going wrong, Claude is guessing at "better" the same way the requester was.; See also: 7.1 Diagnosing an Underperforming Prompt
Real Constraints vs. Generic Improvement Requests: The principle that revising an artifact (a schema, workflow, or document) requires handing Claude the actual rules it must satisfy — what entities exist, how they relate, what must never be null or duplicated — rather than asking generically to "make it better" or "more professional." This pattern generalises across writing, coding, and design troubleshooting alike.; Exam context: Expect scenarios contrasting a vague "improve this" request against one that supplies specific business rules or constraints; the exam favours the constraint-grounded revision every time.; See also: 7.1 Diagnosing an Underperforming Prompt
Consistent Test Set and Success Criteria: A fixed, small list of representative test cases (three to five is often enough) plus a simple, consistently-applied rule for what counts as acceptable, run against both the old and new version of a prompt or workflow before deciding an adjustment helped. Holding the test cases and criteria constant is what allows a performance difference to be attributed to the actual change rather than to a shift in how each version was evaluated.; Exam context: Watch for distractors suggesting you "refresh" or vary test inputs between versions "to keep things realistic" — this breaks the comparison. Test-set changes belong between full evaluation cycles, not mid-comparison, and the standard should never be loosened or tightened for a borderline case partway through.; See also: 7.2 Adjusting Approach Based on Feedback and Results
Anecdotal Evidence Trap: The failure pattern of running an adjusted prompt once against a single input, seeing an output that looks better, and declaring the problem solved. A single good-looking result only shows the change *can* help on that one case — it doesn't show the change *reliably* helps across the range of inputs the workflow needs to handle in practice.; Exam context: A frequent exam scenario: a team concludes an issue is fixed after one better-looking run. The correct objection is always that a single result is weak evidence and a fixed, multi-case comparison is needed — not an unrelated fix like changing models or effort level.; See also: 7.2 Adjusting Approach Based on Feedback and Results
Troubleshooting Decision Checklist: An ordered checklist for diagnosing underperforming output when the cause isn't obvious, run roughly in order because earlier items are cheaper to check: (1) vague instruction missing a role or explicit criteria, (2) inconsistent format or structure despite detailed instructions, (3) a specific claim or number that may simply be wrong, (4) wrong model or effort level for the task's demands, (5) a conversation that has simply gotten too long. Each category points to a distinct fix — role/criteria, worked examples, verification/grounding, model or effort selection, or starting fresh.; Exam context: The exam pairs a scenario describing one specific, checkable failure with a distractor proposing an unrelated fix (e.g. a wrong number paired with "add examples" or "switch models"). The correct answer always targets the checklist category the scenario actually describes, and "start a new conversation" is a late, not first, resort.; See also: 7.3 A Decision Framework for Poor Output
Reusable Projects (Workflow Efficiency): A claude.ai Project that stores the stable context for a recurring task once — a system prompt describing the role and rules, and knowledge base files holding reference material like a style guide or policy document — so every new conversation inside the Project starts with it already in place. Each run then only needs to supply the one thing that actually varies, removing the wasted effort of re-explaining role, rules, and background by hand every time.; Exam context: A common distractor proposes rebuilding the same context from scratch in a fresh, generic conversation each run "to keep things fresh" — that is exactly the repeated effort a Project setup exists to eliminate.; See also: 7.4 Optimising a Workflow for Efficiency
Prompt Templates for Recurring Tasks: A saved, reusable version of a message for a task run often enough to have a stable shape, with clear placeholders for the parts that change. Templates remove the friction of re-deciding how to phrase the request every run and front-load clear, specific instructions that produce good output on the first try, instead of a requester improvising a slightly different, vaguer request each time and needing a follow-up correction.; Exam context: Templates are tested as one of the concrete levers for workflow efficiency, distinct from Projects (which hold context) and model selection (which controls cost/speed per run).; See also: 7.4 Optimising a Workflow for Efficiency
Right-Sizing Model Choice: Matching model selection to the actual stakes of each run of a recurring workflow, rather than defaulting to one setting out of habit. A routine, high-volume, low-stakes run (classifying incoming requests, a first-pass summary a person will review anyway) is a good candidate for a faster, lighter model, since savings compound every run and human review catches occasional misses. A high-stakes run of the same workflow — a client-facing deliverable, a legal or financial document — still warrants the most capable model available.; Exam context: The exam distractor is defaulting to the most capable, most thorough model for every run "to be safe" — this ignores the actual stakes of routine, reviewed runs and pays a real cost every single execution. Efficiency means matching the setting to stakes, not always choosing the cheapest or always the most expensive option.; See also: 7.4 Optimising a Workflow for Efficiency
Front-Loading Instructions: Folding a correction that keeps getting sent by hand after every run ("also include the date," "shorter, please," "don't forget the disclaimer") directly into the reusable Project instructions or prompt template, instead of repeating it each time. A workflow that technically works but needs two or three follow-up corrections on every run isn't actually efficient, even if the final output is fine, because each extra turn costs time and, in an API context, additional tokens.; Exam context: The signal to watch for is a correction typed more than once on the same workflow — the exam treats this as proof the fix belongs in the underlying instructions, not as an acceptable ongoing cost of "only one extra message."; See also: 7.4 Optimising a Workflow for Efficiency