Prompt Engineering & Structured Output · Lesson 3 of 6

4.3 - Structured Output with Tool Use

Enforce structured output using tool use and JSON schemas

When you need guaranteed schema-compliant structured output from Claude, there is a clear reliability hierarchy:

tool_use with JSON schemas - eliminates JSON syntax errors entirely
Prompt-based JSON - model can produce malformed JSON

Commit this hierarchy to memory. The exam builds on it. With tool use, the tool's JSON schema constrains the shape of what Claude returns, eliminating syntax issues like missing brackets, trailing commas, or unquoted keys. The separate tool_choice parameter is what forces the model to call the tool at all. Prompt-based extraction (asking the model to output JSON in a text response) gives you no structural guarantees and will periodically produce unparseable output in production.

tool_choice: The Three Modes

The tool_choice parameter controls whether and how the model calls tools. Understanding the three modes is critical for the exam:

"auto" (default): The model decides whether to call a tool or return text. It may choose to respond with a text message instead of calling the extraction tool. Use this when the model legitimately needs the option to respond conversationally.

"any": The model MUST call a tool but chooses which one. Use this when you have multiple extraction schemas (e.g., extract_invoice, extract_receipt, extract_contract) and the document type is unknown. The model selects the appropriate tool and returns structured output. Guaranteed structured output, flexible tool selection.

{"type": "tool", "name": "extract_metadata"}: The model MUST call the specific named tool. Use this to force a mandatory first step - for example, ensuring metadata extraction runs before enrichment steps. No flexibility, maximum control.

// Force guaranteed structured output with unknown document type
const response = await client.messages.create({
  model: "claude-sonnet-5",
  max_tokens: 4096,
  tool_choice: { type: "any" },
  tools: [extractInvoiceTool, extractReceiptTool, extractContractTool],
  messages: [{ role: "user", content: documentText }]
});

// Force a specific extraction step
const response = await client.messages.create({
  model: "claude-sonnet-5",
  max_tokens: 4096,
  tool_choice: { type: "tool", name: "extract_metadata" },
  tools: [extractMetadataTool],
  messages: [{ role: "user", content: documentText }]
});

What tool_use Does NOT Prevent

This is where the exam gets sneaky. tool_use with JSON schemas eliminates syntax errors but does NOT prevent semantic errors:

Sum discrepancies: Line items that do not sum to the stated total
Field placement errors: Values placed in the wrong fields (e.g., a date in an amount field when both are strings)
Fabrication: The model invents values for required fields when the source document lacks the information

The schema guarantees structure. It doesn't guarantee correctness. Semantic validation needs additional logic (covered in Task Statement 4.4).

Schema Design for Production

Effective schema design prevents entire classes of errors at the structural level:

Optional/nullable fields - When source documents may not contain certain information, make those fields optional or nullable. This is the primary defence against fabrication. If a field is required, the model is pressured to produce a value even when the source has none. If the field is nullable, the model can honestly return null.

{
  "type": "object",
  "properties": {
    "invoice_number": { "type": "string" },
    "vendor_name": { "type": "string" },
    "payment_terms": { "type": ["string", "null"] },
    "purchase_order": { "type": ["string", "null"] }
  },
  "required": ["invoice_number", "vendor_name"]
}

"unclear" enum value - For ambiguous cases where the source is genuinely unclear, add an explicit "unclear" option to enum fields. This prevents the model from forcing a classification when the evidence is ambiguous.

"other" + detail string - For extensible categorisation, include an "other" enum value paired with a freeform detail string field. This captures edge cases that your predefined categories do not cover.

{
  "category": {
    "type": "string",
    "enum": ["invoice", "receipt", "contract", "unclear", "other"]
  },
  "category_detail": {
    "type": ["string", "null"],
    "description": "Freeform detail when category is 'other'"
  }
}

Format normalisation rules - Include format normalisation instructions in the prompt alongside the schema. The schema enforces structure. The prompt enforces formatting consistency (e.g., "All dates in ISO 8601 format," "All currency amounts as decimal numbers without currency symbols").

Key Concept

tool_use with JSON schemas eliminates syntax errors but not semantic errors. Make fields optional/nullable when source documents may lack information - this prevents the model from fabricating values. Use tool_choice "any" for guaranteed structured output when the document type is unknown.

Exam traps

Believing tool_use with JSON schemas prevents all extraction errors

tool_use eliminates JSON syntax errors only. Semantic errors - values that do not sum correctly, data placed in wrong fields, fabricated values for missing information - still occur and require separate validation.
Confusing tool_choice 'auto' with 'any'

'auto' allows the model to return text instead of calling a tool - no guarantee of structured output. 'any' guarantees a tool call but lets the model choose which tool. For guaranteed structured output with unknown document types, use 'any'.
Making all schema fields required to ensure data completeness

Required fields pressure the model to fabricate values when information is absent from the source. Optional/nullable fields allow honest null responses, which is always preferable to plausible-looking fabricated data.

Practice question

Your extraction system uses tool_use with a strict JSON schema where all fields are required. Testers report the model invents plausible-looking dates and monetary amounts when processing documents that lack this information. What is the best fix?

A Add an instruction to the prompt telling the model that it must not hallucinate any values at all
Vague instructions do not override the schema constraint. Required fields structurally pressure the model to produce values regardless of instructions.
B Switch from tool_use to prompt-based JSON extraction, which gives more flexibility in the output
This moves backwards in the reliability hierarchy. Prompt-based JSON introduces syntax errors without solving the fabrication problem.
C Make fields optional or nullable when source documents may not contain the information Correct
Optional/nullable fields allow the model to return null instead of fabricating values. This addresses fabrication at the schema design level - the root cause.
D Add a post-extraction validation step that checks all values against the source document
Post-hoc validation is valuable but addresses symptoms. Making fields optional prevents fabrication at the schema level, which is the correct root cause fix.

Build exercise: Build a Structured Extraction Tool with JSON Schema

Intermediate · 45 minutes

You'll practice:

Design JSON schemas with optional/nullable fields to prevent fabrication of missing data
Understand the three tool_choice modes (auto, any, forced) and when to use each
Recognise that tool_use eliminates syntax errors but not semantic errors
Apply schema design patterns: unclear enum values, other + detail string, format normalisation

Define an extraction tool with a JSON schema: 3 required fields, 3 optional/nullable fields, an enum with unclear and other options, and a detail string field for the other category

Schema design directly prevents fabrication. Required fields pressure the model to invent values when information is absent. Optional/nullable fields allow honest null responses. This is the root cause fix for hallucinated extraction data.

You should see: A valid JSON schema with required array containing only the 3 always-present fields, nullable type definitions for optional fields, and an enum array including unclear and other alongside the standard categories.

Hints

Think about which fields are guaranteed to appear in every document versus which may be absent.

Use the JSON Schema pattern type: ["string", "null"] for nullable fields. Include the category_detail field with a description explaining it is for the other category.

const extractTool = {
  name: "extract_document",
  description: "Extract structured data from a document",
  input_schema: {
    type: "object",
    properties: {
      invoice_number: { type: "string" },
      vendor_name: { type: "string" },
      document_date: { type: "string", description: "ISO 8601 format" },
      payment_terms: { type: ["string", "null"] },
      purchase_order: { type: ["string", "null"] },
      tax_id: { type: ["string", "null"] },
      category: { type: "string", enum: ["invoice", "receipt", "contract", "unclear", "other"] },
      category_detail: { type: ["string", "null"], description: "Detail when category is other" }
    },
    required: ["invoice_number", "vendor_name", "document_date"]
  }
};

Test with tool_choice auto and observe cases where the model returns text instead of calling the tool

The exam tests the distinction between auto, any, and forced tool_choice. Auto allows the model to respond conversationally instead of calling a tool, which means no guaranteed structured output. You need to see this failure mode firsthand.

You should see: At least one response where the model returns a text message describing the document contents instead of calling the extraction tool. This demonstrates why auto is unsuitable when you need guaranteed structured output.

Hints

Try passing a short or ambiguous document to increase the chance the model responds with text rather than calling the tool.

Set tool_choice: { type: "auto" } and send 3-5 different document types. Watch the response stop_reason field: tool_use means the tool was called, end_turn means text was returned.

const response = await client.messages.create({
  model: "claude-sonnet-5",
  max_tokens: 4096,
  tool_choice: { type: "auto" },
  tools: [extractTool],
  messages: [{ role: "user", content: documentText }]
});
console.log("Stop reason:", response.stop_reason);
// If stop_reason is "end_turn", the model returned text, not structured output

Switch to tool_choice any and verify the model always returns structured output via a tool call

tool_choice any guarantees a tool call while letting the model choose which tool. This is the correct setting for guaranteed structured output when the document type is unknown, a key exam distinction from auto.

You should see: Every response has stop_reason of tool_use and contains a valid tool call with structured output conforming to your schema. No text-only responses.

Hints

Run the same documents from the previous step with tool_choice any and compare the stop_reason values.

If you have multiple extraction tools (e.g., extract_invoice, extract_receipt), any lets the model pick the right one. With a single tool, any and forced produce the same result.

const response = await client.messages.create({
  model: "claude-sonnet-5",
  max_tokens: 4096,
  tool_choice: { type: "any" },
  tools: [extractInvoiceTool, extractReceiptTool, extractContractTool],
  messages: [{ role: "user", content: documentText }]
});
// stop_reason should always be "tool_use"
// Check which tool was selected: response.content[0].name

Force a specific tool with tool_choice {type: tool, name: extract_metadata} and verify the mandatory extraction step runs

Forced tool selection ensures a mandatory first step executes regardless of the model decision. The exam tests this for scenarios like metadata extraction that must run before enrichment steps.

You should see: The response always calls the exact tool you specified, even when the document content might suggest a different tool would be more appropriate. The model has no flexibility in tool selection.

Hints

Create a second tool (e.g., extract_metadata) and force it. Send a document that might naturally suit a different tool to confirm forcing overrides model preference.

The format is tool_choice: { type: "tool", name: "extract_metadata" }. This is maximum control mode with no flexibility.

const response = await client.messages.create({
  model: "claude-sonnet-5",
  max_tokens: 4096,
  tool_choice: { type: "tool", name: "extract_metadata" },
  tools: [extractMetadataTool, extractDetailTool],
  messages: [{ role: "user", content: documentText }]
});
// response.content[0].name will always be "extract_metadata"

Process 5 documents - 3 with complete data and 2 with missing fields - and verify nullable fields return null rather than fabricated values

This validates the most important schema design principle: optional/nullable fields prevent fabrication. The exam specifically tests the scenario where required fields pressure the model to invent plausible-looking data for absent information.

You should see: For the 3 complete documents, all fields populated with correct values. For the 2 documents missing information, the nullable fields return null instead of fabricated values. No invented dates, amounts, or identifiers.

Hints

Create test documents where specific fields are genuinely absent, not just hard to find. For example, a receipt with no purchase order number.

Compare the output with all fields required versus your nullable schema. The required version will invent values; the nullable version will return null.

// Document missing payment_terms and tax_id
const incompleteDoc = "Invoice #1234 from Acme Corp, dated 2024-03-15. Total: $500.00";

// With nullable fields, expect:
// payment_terms: null
// tax_id: null
// With all-required schema, the model invents plausible values

const result = response.content[0].input;
console.assert(result.payment_terms === null, "Should be null, not fabricated");
console.assert(result.tax_id === null, "Should be null, not fabricated");

Sources

Claude Certified Architect Foundations Exam Guide - Task Statement 4.3 - Anthropic
Tool Use (Function Calling) - Anthropic
Building with Claude API (Skilljar) - Anthropic