Explicit Criteria: Symptom → Fix
| Symptom | Fix | Why |
|---|---|---|
| System prompt states a goal but no threshold ("be conservative", "flag inappropriate content") | Rewrite as explicit report/skip categories plus a rule for at least one named borderline case | An inferred threshold drifts between similar cases - explicit criteria give the model a decidable rule |
| Findings filtered by the model's self-reported confidence score | Fix the underlying criteria first; use confidence routing only as a secondary layer afterward | LLM self-reported confidence is poorly calibrated - confidently wrong and hesitantly right both happen |
| One noisy, high-false-positive category in a multi-category prompt | Temporarily disable that category while reworking its criteria with concrete examples | Trust isn't scoped per category - a bad category poisons trust in every category from the same prompt |
| Prompt keeps getting longer/more earnest-sounding to fix inconsistency | Add concrete, checkable criteria instead of more descriptive language | Length isn't the lever - a short prompt with a decidable rule beats a long, still-vague one |
Precision vs. Recall Defaults
Every classification-shaped prompt (flagging, extraction, routing, moderation) makes this trade-off whether you name it or not - name it on purpose.
| Bias | Default rule | Good fit |
|---|---|---|
| Precision (fewer false positives) | "When in doubt, don't flag" | Spam filtering - anything where a false alarm wastes reviewer time |
| Recall (fewer missed cases) | "When in doubt, flag for review" | Fraud detection, safety triage - anything where a miss is costly |
Name at least one concrete borderline case in the prompt and state which way to err on it - general wording about being "careful" or "thoughtful" doesn't resolve it.
tool_choice Modes
| Mode | Behavior | Use when |
|---|---|---|
{"type": "auto"} (default) | Model may call a tool or return plain text | Conversational agent - never when structured output must be guaranteed |
{"type": "any"} | Model must call some tool, picks which | Input type unknown ahead of time, multiple candidate schemas |
{"type": "tool", "name": "..."} | Model must call this specific tool | A mandatory step with zero flexibility |
Check stop_reason: tool_use means a tool was called; end_turn under auto means it wasn't - that's the gap auto doesn't close.
Schema Design & Validation Checklist
- Use a tool
input_schema, not a prompt asking for JSON in free text - read the parsed object off thetool_useblock. - Make fields nullable/optional wherever the source may legitimately lack that data; add an
"unclear"/"other"enum escape valve. - Required fields pressure fabrication - a well-typed field can still be an invented value the schema can't catch.
- Validate semantics in your own code (sums, cross-field consistency, enum sanity) - a schema guarantees shape, not correctness.
- On a validation failure, send a specific corrective
tool_result(is_error: true+ the exact problem), not a generic "try again." - Cap retries at 2–3 attempts, then route to human review - unbounded retry on an unsolvable case is just a slower failure.
Few-Shot Prompting Checklist
- Use 3–5 examples - diversity across the real range of cases beats a larger pile of near-identical ones.
- Include at least one edge case (missing field, dual-category input) - exactly where prose-only instructions leave the model guessing.
- Vary incidental surface details (names, numbers) across examples so only the pattern generalizes, not the incidental content.
- Wrap every example and the live input in clear tags (
<example>,<input>) - an unmarked example can be mistaken for real input to act on. - Put the example you most need honored last - examples nearer the end of the block carry more weight, especially when signals conflict.
- Put stable criteria in the system prompt (4.1) and few-shot examples in the user turn near the live input - they solve different problems and stack.
- Distractor to reject: "add more examples" or "remove the examples" when the actual fix is delimiting them clearly.
Context Failure Modes → Fix
| Symptom | Likely cause | Fix |
|---|---|---|
| Long-running agent starts contradicting an early, correctly-established finding | Progressive summarization trap - repeated summary-of-summary compounds information loss | Avoid re-summarizing an already-summarized history repeatedly; summarize from source material when possible |
| A fact stated once early in a long session stops being honored later | Lost-in-the-middle effect - mid-context content is attended to less reliably | Restate genuinely important facts near the end of the context, not just once early on |
input_tokens keeps climbing across turns even though old tool output is no longer needed | Large stale tool results resent unchanged on every call | Trim the specific stale tool result to a short placeholder, not the whole conversation |
Context is trimmed but cache_read_input_tokens drops to near zero | Trimming/rewriting an earlier message invalidated the cache prefix | Trim in batches at natural checkpoints, not every turn; keep the cache breakpoint before the volatile tail |
Trimming Approaches Compared
| Approach | What it targets | Risk |
|---|---|---|
| Naive sliding window (drop oldest N messages) | Age of the message, regardless of content | Can discard a short, still-relevant instruction while leaving a huge stale tool result untouched |
| Targeted stale tool-result trimming | The actual driver of token growth | Low risk if a minimal placeholder is kept so the model still knows the step happened |
| Repeated whole-conversation re-summarization | Entire history compressed into a running summary | Compounding information loss across passes - the progressive summarization trap |
| Per-turn trimming with prompt caching enabled | Immediate token savings on the current call | Invalidates the cache prefix for every turn after the edit, raising cost on the next call |