PPassTheCertExam

Study guides / CCAO-F / Domain 7

Troubleshooting and Optimisation · Lesson 1 of 4

7.1 — Diagnosing an Underperforming Prompt

Use a repeatable try-notice-adjust loop, grounded in real failing examples and the problem's actual constraints, to diagnose why a prompt is underperforming.

Troubleshooting a Claude prompt starts from the same discipline as troubleshooting anything else: you can't fix what you haven't precisely described. "The output isn't very good" isn't a diagnosis — it's a symptom. Diagnosis means running the prompt, collecting specific real outputs you weren't happy with, and naming exactly what's wrong with each one (wrong tone, missed a requirement, invented a fact, inconsistent structure between runs) before you touch the prompt at all.

This matters because the fix for "Claude's tone is off" is often completely different from the fix for "Claude keeps skipping a step," and both are completely different from "Claude's output looks plausible but the numbers are wrong." Skipping straight to a rewrite without first isolating which of these you're actually looking at means you're as likely to make things worse as better.

The Try-Notice-Adjust Loop

Prompt troubleshooting works best as a small, repeatable loop rather than a one-time rewrite: try the prompt against a real input, notice precisely where the output falls short (not just that it's "bad"), and adjust with one targeted change aimed at that specific shortfall. Then run the loop again. Ad-hoc tweaking without running the loop — changing three things at once because the output "felt off" — tends to fix the one case you happened to be staring at while quietly breaking something else you weren't checking.

The discipline here is resisting the urge to rewrite the whole prompt from scratch the moment something looks wrong. A prompt that's 90% working and fails on one category of input needs a targeted diagnosis of that one category, not a ground-up replacement of the 90% that was already fine.

Asking Claude to Diagnose Its Own Failing Prompt

Claude can help with diagnosis directly, and this is one of the most efficient troubleshooting moves available on the exam and in practice: paste in the underperforming prompt along with one or two concrete examples of outputs you weren't happy with, and ask Claude to explain what about the prompt likely caused each specific problem, and suggest targeted fixes. Given real failing examples to reason from, Claude will typically point at the same underlying techniques covered elsewhere in this course — a missing role, missing examples, ambiguous instructions, no explicit criteria — as concrete, applied diagnoses rather than abstract advice.

The trap is asking generically: "here's my prompt, make it better," with no failing example attached. Without a concrete instance of the output actually going wrong, Claude is guessing at what "better" means in exactly the same way you were before you started diagnosing — it has nothing to diagnose against, only a prompt that looks reasonable in isolation.

Feeding Claude the Real Constraints, Not a Generic Ask

The same principle extends past writing tasks. Suppose a basic database schema Claude produced is underperforming — it technically works but doesn't reflect how the business actually operates. Asking Claude to "make the schema better" or "make it more professional" gives it nothing to diagnose against; it will guess at plausible-sounding improvements that may have nothing to do with your actual problem. The stronger move is to hand Claude the real business rules the schema needs to support — what entities exist, how they relate, what must never be null or duplicated — and ask for a revised schema with explicit relationships and constraints derived from those specific rules. Diagnosing against real constraints, not asking for generic improvement, is the pattern that generalises across writing, coding, and design troubleshooting alike.

Key Concept

Diagnose an underperforming prompt with a repeatable try-notice-adjust loop: run it, name precisely what's wrong with the actual output, make one targeted change, and re-run. Give Claude itself the failing examples and the problem's real constraints when asking it to help — not a generic "make this better."

Common Exam Distractor

Watch for answers that describe rewriting a prompt from scratch, or asking for a generic improvement, immediately after noticing one bad output — with no specific failing example and no named cause. The exam favours diagnosing the precise shortfall first (tone? structure? missing step? wrong fact?) and making one targeted change grounded in a real example, over a broad, unfocused rewrite.

Exam traps

Practice question

A team's Claude-generated weekly report keeps missing one specific required section, even though the prompt technically asks for it. The team's first instinct is to discard the current prompt and write an entirely new one from scratch. What is the better troubleshooting approach?

  • A Discard the current prompt and write a completely new one, since the existing prompt is clearly unreliable

    A full rewrite risks losing the parts of the prompt that are already working correctly, and doesn't target the specific, isolated failure — a missing section — that's actually occurring.

  • B Paste the current prompt and a real report output where the section was missing into a conversation with Claude, and ask it to diagnose why that specific section might be getting dropped Correct

    Giving Claude the actual prompt plus a concrete failing example lets it reason about the specific, likely cause — for example, an instruction that's present but ambiguous or easy to deprioritize — rather than guessing at a generic rewrite.

  • C Ask Claude generically to 'make this report prompt better' with no example attached

    Without a concrete failing example, Claude has nothing specific to diagnose against and is likely to suggest broad, unfocused changes unrelated to the actual missing-section problem.

  • D Lower the model's effort level so the report generates faster, then re-check whether the section reappears

    Effort level affects thoroughness and cost, not whether a specific instruction is clear enough to reliably act on; this doesn't address the likely cause of a consistently dropped section.

Build exercise: Run a Diagnosis Loop on a Real Prompt

Beginner · 25 minutes

You'll practice:

  1. In claude.ai, write a short prompt for a task you care about (e.g. drafting a weekly status update from bullet notes) and run it against two different real inputs. For each output, write one precise sentence describing exactly what's wrong with it — not 'not great,' but the specific shortfall (e.g. 'drops the blockers section' or 'tone is too casual for a leadership audience').

    Precise, named shortfalls are what make diagnosis possible. A vague sense that something 'isn't quite right' can't be diagnosed or fixed with a targeted change.

    You should see: Two outputs, each with a one-sentence, specific description of what's actually wrong — not a general impression.

    Hints 
    1. Pick a task where you can judge correctness at a glance, like whether a required section is present.
    2. Force yourself to name one concrete issue per output, even if others exist — pick the most important one.
    3. If both outputs look fine at first glance, try a trickier third input to surface a real gap.

  2. In a new message, paste in your original prompt, one of the outputs, and your one-sentence description of what was wrong. Ask Claude to explain the likely cause in the prompt and suggest one targeted fix.

    This uses Claude itself for diagnosis, grounded in a real failing example — the reliable version of 'ask Claude to make this better.'

    You should see: A specific diagnosis (e.g. 'the instruction to include blockers is present but buried after several other requests') and one concrete suggested change, not a full rewrite.

    Hints 
    1. If Claude's response is a full prompt rewrite rather than a targeted diagnosis, ask it to first explain the cause before suggesting the fix.
    2. Apply only the one targeted change it suggests, not every idea it mentions.
    3. Re-run the revised prompt against the same input and confirm the specific issue is resolved.

  3. Pick a second, non-writing task you could ask Claude to help design (a simple form, a checklist, a basic categorisation scheme) and, instead of asking generically for 'a better version,' write down two or three real constraints it needs to satisfy, then ask Claude to revise it against those specific constraints.

    This confirms the pattern generalises: feeding Claude real constraints produces a grounded revision, while a generic 'make it better' request does not, regardless of task type.

    You should see: A revised version that visibly reflects the specific constraints you supplied — not just a stylistically different version of the original.

    Hints 
    1. Constraints can be simple: 'every entry needs a due date' or 'no category can overlap with another.'
    2. Compare this result to what you'd expect from asking generically for 'something better,' with no constraints listed.
    3. Write one sentence on which version you'd actually trust to use — and why.

Sources