Troubleshooting and Optimisation · Lesson 3 of 4

7.3 — A Decision Framework for Poor Output

Work through a systematic checklist of likely causes — vague instructions, missing examples, unverified facts, wrong model or effort level, or an overlong conversation — before deciding how to fix underperforming output.

By this point in the course you've seen several distinct techniques for improving Claude's output, spread across different domains: roles and explicit criteria, worked examples, verification and citation habits, model and effort selection, and conversation management. When output is actually bad in front of you, though, the hard part usually isn't knowing any one of these techniques — it's knowing which one applies to the specific way this particular output is failing. This lesson ties those techniques together into one ordered checklist you can run through whenever output is underperforming and it isn't immediately obvious why.

The Checklist

Work through these questions roughly in order, since earlier ones are cheaper to check and rule out than later ones:

Is the instruction vague or missing a role and explicit criteria? If output is generically toneless, inconsistent, or seems to be guessing at an unstated standard, the fix is usually a concrete role and explicit rules, not more prose describing the desired quality in the abstract.
Is the format or structure inconsistent across runs, even though the instructions are already fairly detailed? This points to needing worked examples rather than more written instruction — showing the exact pattern is more reliable than describing it.
Could a specific claim or number in the output simply be wrong? Any output stating facts, figures, or citations needs to be checked against a source, or the prompt needs to explicitly ask Claude to ground its answer in provided material and flag uncertainty, rather than being trusted at face value.
Is this the wrong model or effort level for how demanding the task actually is? A fast, lightweight model or a low effort setting can under-deliver on a task that's genuinely complex or high-stakes; conversely, a simple, high-volume task doesn't need the most capable, most thorough setting available.
Has the conversation simply gotten too long? A long-running conversation can drift, lose track of earlier instructions, or start blending unrelated context together. If none of the above explain the problem, and the conversation has been going for a long time, starting fresh with a clean, well-structured prompt is often the fix, rather than continuing to patch an overloaded thread.

Worked Example: A Weekly Sales Summary Gone Wrong

Suppose a Claude-generated weekly sales summary states a growth percentage that turns out to be incorrect when checked against the source spreadsheet, while everything else about the summary — tone, structure, length — looks fine. Running the checklist: the instruction isn't vague (the summary is well-organized), so that's not it. The format is consistent across weeks, so it's not a missing-examples problem. The issue is a specific, checkable number being wrong — that lands squarely on the third checklist item. The fix isn't a new role or more examples; it's asking Claude to show its calculation from the source figures explicitly, or having a person spot-check the specific number before the summary goes out, the same verification discipline covered when working with facts and citations elsewhere in this course. Model or effort level and conversation length were never the actual cause here, even though they're plausible-sounding guesses if you didn't work through the checklist in order.

Key Concept

When output is underperforming and the cause isn't obvious, work through a checklist in order: vague instruction (role/criteria) → inconsistent format (examples) → possibly wrong fact (verify/ground) → wrong model or effort level for the task → conversation grown too long (start fresh). Matching the fix to the actual cause, rather than guessing, is far more reliable than reaching for whichever technique you tried most recently.

Common Exam Distractor

A scenario describing one specific, checkable failure (like a wrong number) often comes paired with a distractor answer proposing an unrelated fix that sounds generically reasonable — switching models, adding examples, rewriting the role. If the scenario clearly identifies which checklist category the problem belongs to, the correct answer targets that category specifically, not a plausible-sounding technique borrowed from a different one.

Exam traps

Applying a role/examples fix to a problem that's actually a factual-accuracy issue

A well-structured, consistently-toned output can still contain a wrong number or claim; role and example fixes address tone and format, not factual grounding — they won't catch or prevent an incorrect figure.
Assuming any underperforming output means the wrong model or effort level was used

Model and effort level are only the right diagnosis when the task's complexity genuinely exceeds what the current model/effort setting can handle — many quality problems are prompt-structure or verification issues that a bigger model won't fix.
Jumping to 'start a new conversation' as a first response to any bad output

Conversation length is a late-checklist item for a reason — most bad output has a more specific, cheaper-to-fix cause (vague instruction, missing examples, unverified fact, mismatched model) that starting fresh won't address if it recurs in the new conversation too.

Practice question

A Claude-generated weekly sales summary is well-organized and consistently formatted week to week, but this week it states a growth percentage that turns out to be wrong when checked against the source spreadsheet. Using a systematic troubleshooting checklist, what is the most likely cause and appropriate fix?

A The instructions are too vague; add a specific role and explicit formatting criteria
The summary is already well-organized and consistently formatted, which rules out a vague-instruction or missing-role problem — that's not what's failing here.
B The output format is inconsistent; add three to five worked examples
Format and structure are described as consistent across weeks, so inconsistent formatting isn't the actual problem being described.
C A specific factual claim (the growth percentage) may simply be wrong; verify it against the source figures or have Claude show its calculation explicitly before trusting it Correct
This is a checkable, specific factual error rather than a tone, structure, model-capability, or conversation-length problem. Verifying the number against the source, or asking Claude to show its work, directly targets the actual cause.
D The conversation has likely gone on too long; start a brand-new conversation
Nothing in the scenario indicates conversation drift, and jumping to a fresh conversation wouldn't address the actual cause — a specific, checkable number being incorrect — which would likely recur without a verification step.

Build exercise: Run the Diagnostic Checklist on a Real Bad Output

Intermediate · 25 minutes

You'll practice:

Apply an ordered checklist to diagnose the specific cause of underperforming output
Distinguish which prior-domain technique actually applies to a given failure
Practice ruling out categories before selecting a fix

Find or recreate one real example of Claude output that disappointed you (or deliberately produce one — e.g. ask Claude a question that requires a specific calculated figure from a document you provide, with a vague, unstructured prompt). Write down, in one sentence per checklist item, whether each of the five checklist categories (vague instruction, inconsistent format, possibly-wrong fact, wrong model/effort, overlong conversation) plausibly applies.

Working through every category explicitly, even the ones that obviously don't apply, is what prevents jumping to a plausible-sounding but wrong diagnosis.

You should see: A short five-line list where most categories are ruled out with a one-sentence reason, and one or two categories are flagged as the likely actual cause.
Hints
1. ```
Be honest about ruling categories out — 'the format is fine, this isn't it' is a useful, valid line.
```
2. ```
If more than two categories seem to apply, you likely have more than one distinct problem stacked together — note both.
```
3. ```
Use a real example if you have one; a manufactured one works too as long as you can check the correct answer.
```
Apply the single fix that matches the category you identified as the actual cause (e.g. if it's a factual-accuracy issue, ask Claude to show its calculation or cite the specific source line it used) and re-run the task.

This confirms the diagnosis was correct by checking whether the targeted fix actually resolves the specific problem — the real test of a diagnosis, not just naming it.

You should see: An output where the specific problem you identified is resolved, without unrelated aspects of the output (tone, format) changing unexpectedly.
Hints
1. ```
Apply only the one fix tied to your diagnosed category — resist also changing the role or adding examples 'just in case.'
```
2. ```
If the problem persists after the targeted fix, that's a sign your diagnosis was wrong — go back to the checklist rather than stacking on more changes.
```
3. ```
Compare the new output specifically against the original failure, not just against a general sense of quality.
```
Write one sentence explaining which distractor category you might have wrongly reached for if you hadn't worked through the checklist in order — and why it wouldn't have actually fixed the problem.

This directly rehearses the exam's distractor pattern: a plausible-sounding but mismatched fix that doesn't target the real cause.

You should see: A short written note naming a specific wrong-but-tempting fix and explaining concretely why it would have missed the actual issue.
Hints
1. ```
Pick the distractor that would have sounded most reasonable to someone who hadn't diagnosed carefully.
```
2. ```
Ground your explanation in what that wrong fix actually changes versus what was actually broken.
```
3. ```
This is good practice for spotting the same pattern in a multiple-choice question.
```

Sources

Written fresh — synthesises techniques from earlier domains into a troubleshooting checklist
Anthropic docs — Prompt Engineering Overview
Anthropic docs — Models Overview