Prompt & Context Engineering · Lesson 3 of 4

4.3 - Few-Shot Prompting for Output Consistency

Use concrete examples to pin down a format or tone that's hard to fully specify in words alone.

A few well-chosen input/output examples in the prompt often lock down a format or style more reliably than a longer prose description of the same thing — especially for output shapes that are easy to demonstrate but awkward to describe precisely: a specific tone, a particular level of terseness, a citation style, or a formatting convention with edge cases (how to handle a null field, how to abbreviate a long name, what to do when two categories both apply). Zero-shot prose instructions ask the model to derive the pattern from a description; few-shot examples just show the pattern directly, which is a shorter inferential leap and produces less variance across calls.

Choosing examples deliberately

Three to five examples is typically the useful range for locking in a format — enough to establish the pattern without bloating every request's token count. What matters more than the count is coverage: examples that span the real range of cases teach more than the same number of near-identical easy examples. Concretely, a good few-shot set usually includes:

One or two typical, unambiguous cases that establish the baseline pattern.
At least one edge case — a missing field, an unusually long input, a case where two categories could plausibly both apply — because this is exactly where a prose-only instruction tends to leave the model guessing.
Diverse surface details (different names, numbers, subjects) across examples, so the model generalises the pattern rather than partially memorising incidental content from one example and echoing it into outputs where it doesn't belong.

Delimiting examples from the live input

Wrapping each example clearly — XML-style tags are the standard convention Claude is tuned to recognise — keeps the model from confusing a demonstration with the actual current input, which matters most once a real user query happens to resemble one of the examples closely:

<examples>
  <example>
    <input>Order #4291 hasn't shipped in 2 weeks</input>
    <output>{"category": "shipping_delay", "urgency": "high"}</output>
  </example>
  <example>
    <input>Love the new packaging design!</input>
    <output>{"category": "feedback", "urgency": "low"}</output>
  </example>
</examples>

<input>{{live customer message}}</input>

Without that boundary, an unmarked example sitting in the same block of text as the real task reads to the model as more input to act on, not a demonstration of how to act — it might respond to the example message instead of, or in addition to, the real one.

Combining few-shot with explicit criteria

Few-shot examples and explicit criteria (4.1) solve different problems and stack well together: criteria state the rule in words ("skip style preferences, report bugs and security issues"), examples show what applying that rule actually looks like on real input. Criteria alone can still leave format ambiguous; examples alone can still leave the underlying rule under-specified for cases the examples didn't cover. Put the criteria in the system prompt as the stable, reusable rule, and put the few-shot examples in the user turn near the live input — that keeps the examples close to what they're demonstrating while the system prompt stays a clean, cacheable description of the task.

Order matters: recency effects in example sets

Examples nearer the end of a few-shot block tend to weigh more heavily on the model's output than examples earlier in the same block, particularly when the examples send inconsistent signals. Two practical consequences: don't rely on an early example to override a pattern established by later ones, and when one example is the edge case you most need the model to get right, put it last rather than burying it first. If you're rotating or randomly sampling which examples appear in a prompt across calls, be aware that changing which example lands last will shift observed behaviour even when the total example set is unchanged.

Common exam distractor

An unmarked example sitting in the same block of text as the real task can get treated by the model as part of the actual input rather than a demonstration — tag examples clearly and separately from the live request. "Add more examples" is also a common wrong answer when the actual problem is a lack of delimiting or a lack of diversity among the examples already present; quantity doesn't fix an ambiguity problem that's structural.

Key concept

Few-shot examples work by demonstration, not description — they're most valuable exactly where prose is weakest: pinning down format, tone, and edge-case handling that are easy to show and hard to fully specify in words. Delimit clearly, cover the real range of cases including at least one edge case, and place the case you most need honoured last.

Exam traps

Including few-shot examples without clearly delimiting them from the actual user input.

Without clear separation (e.g. XML tags), the model can mistake a demonstration example for real input it's meant to act on, especially when the real query happens to resemble one of the examples.
Assuming more near-identical examples improve consistency more than fewer, more diverse ones.

Three to five examples spanning the real range of cases - including at least one edge case - teach the model more than a larger set of easy, near-identical examples. Coverage matters more than count.
Reusing the same incidental names, numbers, or entities across every few-shot example.

The model can partially generalise the incidental surface details along with the intended pattern, and echo them into outputs where they don't belong. Vary the surface details across examples so only the actual pattern generalises.
Treating example order within a few-shot block as irrelevant to the output.

Examples nearer the end of a few-shot set tend to carry more weight than earlier ones, especially when examples conflict. The example you most need honoured - often the edge case - should generally be placed last, not first.

Practice question

A prompt includes three example customer messages with model responses, followed directly by the real customer's message, all as plain unlabelled text. Occasionally Claude responds to one of the example messages instead of the real one. What's the fix?

A Remove the examples entirely and rely on prose instructions only.
This gives up the consistency benefit few-shot examples provide for a format/tone that's hard to fully specify in prose - the actual fix is clearer delimiting, not removal.
B Wrap each example and the real input in clearly labelled tags so the model can distinguish demonstration from live task. Correct
This directly addresses the confusion - clear delimiting is exactly what prevents an example from being mistaken for the actual input.
C Increase the number of examples from three to ten.
More unlabelled examples would likely make the confusion worse, not better - the problem is the lack of delimiting, not the quantity.
D Move the examples into the system prompt instead of the user message.
Relocating without adding clear delimiting doesn't resolve the core ambiguity between example and live input.

Build exercise: Add tagged few-shot examples to a formatting task

Beginner · 30 minutes

You'll practice:

Write clearly delimited few-shot examples in a real API call
Include an edge case among the examples and verify it transfers to a new input
Observe how example order affects output on conflicting signals

Pick a task with a specific output format (e.g. converting free text into a terse one-line ticket summary with a fixed JSON shape), write two typical examples and one edge-case example, each wrapped in clear <example> tags, followed by a real input in its own <input> tag.

Including the edge case is what actually extends the model's behaviour past the easy, obvious pattern, and tagging is what keeps the real input from being confused with the demonstrations.

You should see: A prompt string with three clearly delimited <example> blocks (two typical, one edge case such as a message with no clear category) followed by a separately tagged live <input>.

Hints

What's genuinely hard about your chosen format that a prose description alone would leave ambiguous? Make that the edge case.

Use the same tag names consistently across every example and the live input - <example><input>...</input><output>...</output></example>, then a final bare <input> for the real one.

const prompt = `<examples>
  <example><input>Order #4291 hasn't shipped in 2 weeks</input><output>{"category":"shipping_delay","urgency":"high"}</output></example>
  <example><input>Love the new packaging!</input><output>{"category":"feedback","urgency":"low"}</output></example>
  <example><input>Charged twice AND item never arrived</input><output>{"category":"billing_and_shipping","urgency":"high"}</output></example>
</examples>

<input>${liveMessage}</input>`;

Send the tagged prompt to the Messages API and verify the real input's output follows the demonstrated format, including correctly handling whatever made the edge-case example tricky.

This confirms the examples actually transferred the pattern rather than just sitting in the prompt unused.

You should see: A response matching the exact JSON shape from the examples, and correct handling of a live input deliberately designed to hit the same tricky case as your edge-case example (e.g. a message that spans two categories).

Hints

Design your live test input to resemble the edge case's structure (e.g. also spanning two categories) but with different surface details, so you're testing pattern transfer, not memorisation.

Parse the response as JSON and check it has the same keys as your examples' output - a shape mismatch means the examples didn't lock in the format.

const res = await client.messages.create({ model: "claude-sonnet-5", max_tokens: 200, messages: [{ role: "user", content: prompt }] });
const parsed = JSON.parse(res.content[0].text);
console.log(Object.keys(parsed)); // should match example output keys exactly

Remove the XML tags (leave the same three examples and real input as plain, unlabelled concatenated text) and re-run with a live input that closely resembles one of your examples in wording.

This reproduces the failure mode from the practice question directly - an unmarked example can get treated as more input to act on rather than a demonstration.

You should see: A noticeably higher chance the model responds to (or blends in) one of the example messages instead of cleanly answering only the live input, compared to the tagged version in step 2.

Hints

What happens if your live input uses very similar wording to one of the examples, with no tags to mark the boundary?

Strip every <example>, <input>, and <output> tag but keep the text content and ordering identical, then compare the response to the same live input from step 2.

const untaggedPrompt = examples.map(e => `${e.input}\n${e.output}`).join("\n\n") + `\n\n${liveMessage}`;
const res = await client.messages.create({ model: "claude-sonnet-5", max_tokens: 200, messages: [{ role: "user", content: untaggedPrompt }] });
// compare res.content[0].text against the tagged version's output

Reorder the three examples so the edge case appears first instead of last, keep everything else identical, and compare the live input's output against the version where the edge case was last.

This tests the recency-weighting nuance directly - whether moving the edge case earlier in the block changes how strongly its handling transfers to the live input.

You should see: Some observable difference in how strongly the edge-case pattern is applied to the live input depending on position - most reliably, the edge-case handling should show up more consistently when it's placed last.

Hints

You're testing a positional effect, so keep every other variable (example content, live input, model, temperature) fixed across both runs.

Run the same live input against both orderings 2-3 times each and compare how often the edge-case pattern (e.g. the dual-category handling) actually shows up in the output.

const orderings = { edgeLast: [typical1, typical2, edgeCase], edgeFirst: [edgeCase, typical1, typical2] };
for (const [name, ex] of Object.entries(orderings)) {
  const p = buildTaggedPrompt(ex, liveMessage);
  const res = await client.messages.create({ model: "claude-sonnet-5", max_tokens: 200, messages: [{ role: "user", content: p }] });
  console.log(name, res.content[0].text);
}

Sources

Reused/adapted from CCAR-F lesson 4-2-few-shot-prompting