Study guides / CCAR-F / Domain 4

Prompt Engineering & Structured Output · Lesson 4 of 6

4.4 - Validation, Retry, and Feedback Loops

Implement validation, retry, and feedback loops for extraction quality

Production extraction systems fail. Documents arrive in unexpected formats, numerical values don't add up, and fields end up in the wrong places. The question isn't whether failures happen but how your system responds. The validation-retry pattern turns those failures into self-correcting workflows.

Retry-with-Error-Feedback

The correct retry pattern sends three pieces of information back to the model:

  1. The original document - so the model has the source to re-examine
  2. The failed extraction - so the model can see what it produced
  3. The specific validation error - so the model knows exactly what went wrong
// Retry with error feedback
const retryMessages = [
  {
    role: "user",
    content: `Original document:\n${originalDocument}\n\n` +
      `Your extraction:\n${JSON.stringify(failedExtraction)}\n\n` +
      `Validation error: Line items sum to £450 but stated_total is £500. ` +
      `Please re-extract, ensuring all line items are captured.`
  }
];

This beats a naive retry by a wide margin. Without the specific error, the model has no guidance for what to fix and usually reproduces the same mistake. With it, the model can target its self-correction: re-examining the document for missed line items, checking field placement, recalculating the total.

The Retry Effectiveness Boundary

This is the concept the exam tests most aggressively in this task statement. Retries have a clear effectiveness boundary:

Retries ARE effective for:

Retries are NOT effective for:

The exam presents both scenarios and expects you to identify which is fixable. If a document genuinely doesn't contain a department name, no amount of retrying will produce a correct value. Flag the extraction for human review, or return null if the schema allows it.

Self-Correction Flow Design

Rather than relying solely on external validation logic, you can build self-correction into the extraction schema itself:

calculated_total vs stated_total: Extract both the sum the model calculates from individual line items and the total stated in the document. When these differ, you have an automatic discrepancy flag without external logic.

{
  "line_items": [
    { "description": "Widget A", "amount": 150.00 },
    { "description": "Widget B", "amount": 300.00 }
  ],
  "calculated_total": 450.00,
  "stated_total": 500.00,
  "total_discrepancy": true
}

conflict_detected booleans: Add boolean fields that flag when the source document contains contradictory information. For example, if a document states "payment due: 30 days" in one section but "payment terms: net 60" in another, the model should extract both and set conflict_detected: true rather than silently picking one.

detected_pattern Fields

For code review and analysis pipelines, add detected_pattern fields to structured findings. This tracks which specific code construct triggered each finding.

{
  "finding": "Potential SQL injection vulnerability",
  "severity": "critical",
  "detected_pattern": "string concatenation in SQL query",
  "file": "user_service.py",
  "line": 42
}

When developers dismiss findings, you can analyse dismissal patterns by detected_pattern. If developers consistently dismiss findings triggered by "variable shadowing in nested scope," that pattern likely needs prompt refinement. This creates a systematic improvement loop: extract, validate, collect dismissal data, refine prompts, repeat.

Schema Syntax Errors vs Semantic Validation Errors

The exam distinguishes between these two error categories:

Schema syntax errors - Malformed JSON, missing required fields, wrong data types. Eliminated entirely by tool_use with JSON schemas (covered in Task Statement 4.3).

Semantic validation errors - Correct JSON structure but incorrect values. Line items that do not sum, dates that precede each other incorrectly, values in wrong fields. These require validation logic outside the schema and are the focus of retry loops.

The overlap between these task statements is intentional. The exam tests whether you understand that tool_use solves the first category but not the second.

Pydantic as the Validation Layer

The exam guide names Pydantic alongside JSON Schema in its hands-on exercise for this task statement: "when Pydantic or JSON schema validation fails, send a follow-up request including the document, the failed extraction, and the specific validation error." In a Python pipeline, Pydantic is the layer that turns "validation failed" into the specific, per-field error messages the retry loop needs.

A Pydantic model does two jobs at once. Parsing enforces structure - types, required fields, enums. Validators enforce semantics - the rules a JSON schema cannot express, like cross-field arithmetic or date ordering. Both failure kinds surface through one ValidationError, with machine-readable errors naming the field and the broken rule:

import json
from pydantic import BaseModel, ValidationError, model_validator

class LineItem(BaseModel):
    description: str
    amount: float

class Invoice(BaseModel):
    line_items: list[LineItem]
    stated_total: float

    @model_validator(mode="after")
    def totals_must_match(self):
        calculated = round(sum(i.amount for i in self.line_items), 2)
        if abs(calculated - self.stated_total) > 0.01:
            raise ValueError(
                f"line items sum to {calculated} but stated_total is {self.stated_total}"
            )
        return self

try:
    invoice = Invoice.model_validate(tool_input)  # the tool_use input from the response
except ValidationError as e:
    errors = "\n".join(
        f"{'.'.join(map(str, err['loc'])) or 'invoice'}: {err['msg']}" for err in e.errors()
    )
    retry_message = (
        f"Original document:\n{original_document}\n\n"
        f"Your extraction:\n{json.dumps(tool_input)}\n\n"
        f"Validation errors:\n{errors}\n\n"
        f"Please re-extract, fixing the identified errors."
    )

The except branch is the retry-with-error-feedback pattern from the top of this lesson - Pydantic simply supplies the third ingredient (the specific error) in a form you can format straight into the prompt. The retry-effectiveness boundary applies unchanged: a validator that fails because information is absent from the source document still means human review, not a retry.

Current state: SDK-level parsing and strict tool use

As of July 2026, the Python SDK's client.messages.parse(..., output_format=Invoice) returns validated Pydantic instances via parsed_output, and strict: true on a tool definition guarantees schema-conformant inputs server-side (Structured Outputs). Neither removes the semantic layer: the platform enforces the schema, your validators enforce the business rules, and the retry loop consumes whichever one fails.

Key Concept

Retry-with-error-feedback works by sending the original document, the failed extraction, and the specific validation error. Retries fix format and structural errors but cannot create information absent from the source document. Always identify whether a failure is fixable before retrying.

Exam traps

Practice question

Your extraction pipeline validates that line item amounts sum to the stated total. For Document A, the calculated sum is £450 but the stated total is £500. For Document B, the 'department' field is missing entirely from the source text. Which retry strategy is correct?

  • A Retry both documents with the validation errors, instructing the model to re-extract all fields

    Document B cannot be fixed by retrying - the department information does not exist in the source. Retrying wastes tokens and will likely produce a fabricated value.

  • B Skip retries for both documents and flag them all for human review to ensure accuracy

    Document A has a likely fixable discrepancy. Skipping the retry wastes the model’s self-correction capability for errors it can actually fix.

  • C Retry both documents with the same prompt, since extraction is non-deterministic and may succeed on a second attempt

    Non-determinism does not create information that does not exist. Document A may benefit from targeted retry; Document B will not benefit regardless of how many attempts you make.

  • D Retry Document A with the discrepancy error; flag Document B for human review since the information is absent from the source Correct

    Document A has a fixable discrepancy - the model likely missed a line item. Document B has genuinely absent information, so retries are ineffective. Flag it for human review or accept null.

Build exercise: Build a Validation-Retry Loop for Document Extraction

Advanced · 60 minutes

You'll practice:

  1. Define an extraction tool with calculated_total and stated_total fields, a conflict_detected boolean, and detected_pattern fields for tracking which constructs trigger findings

    Self-correction fields like calculated_total vs stated_total enable automatic discrepancy detection without external logic. conflict_detected booleans and detected_pattern fields create the data foundation for systematic prompt improvement.

    You should see: A JSON schema with separate calculated_total and stated_total number fields, a total_discrepancy boolean, a conflict_detected boolean, and a detected_pattern string field on each finding in the line_items array.

    Hints
    1. Think of the schema as having two layers: the extraction data and the self-assessment metadata.
    2. Include both totals as separate required numeric fields. Add detected_pattern as a string describing the code construct or document feature that triggered each finding.
    3. const extractionTool = {
        name: "extract_invoice",
        input_schema: {
          type: "object",
          properties: {
            line_items: {
              type: "array",
              items: {
                type: "object",
                properties: {
                  description: { type: "string" },
                  amount: { type: "number" },
                  detected_pattern: { type: "string" }
                }
              }
            },
            calculated_total: { type: "number" },
            stated_total: { type: "number" },
            total_discrepancy: { type: "boolean" },
            conflict_detected: { type: "boolean" }
          },
          required: ["line_items", "calculated_total", "stated_total", "total_discrepancy"]
        }
      };
  2. Implement validation logic that checks: field completeness, numerical consistency (calculated sum matches stated total), enum validity, and date ordering

    Semantic validation catches errors that tool_use cannot. The exam distinguishes schema syntax errors (eliminated by tool_use) from semantic errors (wrong sums, misplaced values) that require validation logic and retry loops.

    You should see: A validation function that returns an array of specific, actionable error messages. Each error should state what was expected versus what was found, not just that validation failed.

    Hints
    1. Return specific error messages like line items sum to 450 but stated_total is 500 rather than generic validation failed strings.
    2. Check four categories: required fields present, numerical sums match, enum values are valid members, and dates follow logical ordering (start before end, invoice before due date).
    3. function validateExtraction(result) {
        const errors = [];
        const calculatedSum = result.line_items.reduce((sum, item) => sum + item.amount, 0);
        if (Math.abs(calculatedSum - result.stated_total) > 0.01) {
          errors.push(`Line items sum to ${calculatedSum} but stated_total is ${result.stated_total}`);
        }
        if (result.total_discrepancy !== (calculatedSum !== result.stated_total)) {
          errors.push("total_discrepancy flag does not match actual discrepancy state");
        }
        return errors;
      }
  3. Build the retry loop: on validation failure, construct a follow-up message containing the original document, the failed extraction, and the specific validation error

    Retry-with-error-feedback is dramatically more effective than naive retries. Without the specific error, the model has no guidance and typically reproduces the same mistake. With the error, the model can target its self-correction.

    You should see: A retry message that includes all three elements: the original document text, the JSON of the failed extraction, and the specific validation error string. The model should produce a corrected extraction on retry.

    Hints
    1. Structure the retry message with clear sections: Original document, Your extraction, and Validation error so the model can easily parse each component.
    2. Include a maximum retry count (2-3 attempts) to prevent infinite loops on genuinely unfixable errors.
    3. async function retryExtraction(originalDoc, failedResult, errors, maxRetries = 3) {
        let attempt = 0;
        let result = failedResult;
        while (errors.length > 0 && attempt < maxRetries) {
          const retryMessage = `Original document:\n${originalDoc}\n\n` +
            `Your extraction:\n${JSON.stringify(result, null, 2)}\n\n` +
            `Validation errors:\n${errors.join("\n")}\n\n` +
            `Please re-extract, fixing the identified errors.`;
          result = await extract(retryMessage);
          errors = validateExtraction(result);
          attempt++;
        }
        return { result, errors, attempts: attempt };
      }
  4. Test with 5 documents: 2 with fixable errors (misplaced values, wrong totals) and 3 with unfixable errors (absent information) - verify the loop retries only fixable cases

    The retry effectiveness boundary is the most aggressively tested concept in this task statement. Retries fix format mismatches and structural errors but cannot create information absent from the source. The exam presents both scenarios and expects you to identify which is fixable.

    You should see: The 2 fixable documents succeed after 1-2 retries with corrected totals or field placements. The 3 unfixable documents are correctly identified as having absent information and flagged for human review rather than retried.

    Hints
    1. Create documents where information is genuinely absent, not just hard to find. A receipt with no vendor tax ID is unfixable. A receipt where the model miscounted line items is fixable.
    2. Add a classification step before retrying: check whether the validation error references data that exists in the source document. If not, flag for human review immediately.
    3. function isFixableError(error, originalDoc) {
        // Fixable: numerical discrepancies, misplaced values
        if (error.includes("sum to") || error.includes("misplaced")) return true;
        // Unfixable: field references data not in document
        if (error.includes("not found in source")) return false;
        return false; // Default conservative
      }
      
      const fixableErrors = errors.filter(e => isFixableError(e, originalDoc));
      const unfixableErrors = errors.filter(e => !isFixableError(e, originalDoc));
      if (unfixableErrors.length > 0) flagForHumanReview(result, unfixableErrors);
  5. Log detected_pattern data for each finding and analyse which patterns are most frequently dismissed to identify prompt refinement priorities

    detected_pattern fields create a systematic improvement loop. When developers consistently dismiss findings triggered by a specific pattern, that pattern likely needs prompt refinement. This turns dismissal data into actionable prompt improvement priorities.

    You should see: A log or table showing each detected_pattern, its frequency, its dismissal rate, and a prioritised list of patterns needing prompt refinement. Patterns with high dismissal rates should be at the top.

    Hints
    1. Group findings by detected_pattern value and calculate the dismissal rate for each group.
    2. Focus on patterns with both high frequency and high dismissal rates. A rare pattern with high dismissal may not be worth optimising, but a frequent pattern with high dismissal is a priority.
    3. const patternStats = {};
      for (const finding of allFindings) {
        const p = finding.detected_pattern;
        if (!patternStats[p]) patternStats[p] = { total: 0, dismissed: 0 };
        patternStats[p].total++;
        if (finding.wasDismissed) patternStats[p].dismissed++;
      }
      
      // Prioritise by impact: frequency * dismissal rate
      const priorities = Object.entries(patternStats)
        .map(([pattern, stats]) => ({
          pattern,
          dismissalRate: stats.dismissed / stats.total,
          impact: stats.total * (stats.dismissed / stats.total)
        }))
        .sort((a, b) => b.impact - a.impact);

Sources