5.6 - Information Provenance & Multi-Source Synthesis

Build exercise: Build a Provenance-Preserving Synthesis Pipeline

Advanced · 60 minutes

You'll practice:

• Design structured claim-source mappings with claim, source URL, document name, excerpt, and publication date
• Preserve attribution through multi-step synthesis pipelines without loss during summarisation
• Handle conflicting sources by annotating both values rather than arbitrarily selecting one
• Use temporal awareness (publication dates) to distinguish trends from contradictions
• Apply content-appropriate rendering: tables for financial data, prose for news, lists for technical findings

1. Define a structured claim-source mapping schema with fields: claim, sourceUrl, documentName, relevantExcerpt, publicationDate

   Every finding in a multi-agent research system must carry its provenance. Without structured claim-source mappings, attribution dies during summarisation and the final output becomes untraceable plausible-sounding text with no verifiable sources.

   You should see: A TypeScript interface or JSON schema with all five required fields: claim (the assertion), sourceUrl (where found), documentName (title), relevantExcerpt (supporting passage), and publicationDate (when published or data collected). Each field should be required, not optional.

   Hints

   1. The publication date is not just metadata tidiness. It is essential for correct interpretation when two sources report different numbers.

   2. Include a description for each field explaining what it contains and why it is required for provenance tracking.

   3. interface ClaimSourceMapping {
        claim: string;          // The specific assertion being made
        sourceUrl: string;      // Where the information was found
        documentName: string;   // Title of the source document
        relevantExcerpt: string; // Specific passage supporting the claim
        publicationDate: string; // ISO 8601 date of publication or data collection
      }
      
      // Example
      const finding: ClaimSourceMapping = {
        claim: "Global renewable energy investment reached $495 billion in 2023",
        sourceUrl: "https://example.com/iea-report-2024",
        documentName: "IEA World Energy Investment Report 2024",
        relevantExcerpt: "Total investment in renewable energy technologies reached approximately $495 billion in calendar year 2023.",
        publicationDate: "2024-06-15"
      };

2. Implement two research subagents that output findings using the claim-source mapping schema, including publication dates

   Subagents must output in the structured format from the start. If subagents return unstructured prose, attribution is already lost before synthesis begins. Requiring structured output at the subagent level is the foundation of end-to-end provenance.

   You should see: Two subagent functions that each return an array of ClaimSourceMapping objects with all fields populated, including publication dates. Each subagent should research a different aspect of the same topic.

   Hints

   1. Assign each subagent a specific research focus (e.g., market data vs regulatory developments) to simulate a real multi-agent research pipeline.

   2. The subagent prompt must explicitly require structured output with all five claim-source mapping fields. Without this instruction, subagents will return prose.

   3. async function researchSubagent(topic, focus) {
        const response = await client.messages.create({
          model: "claude-sonnet-5",
          max_tokens: 4096,
          tools: [{
            name: "report_findings",
            description: "Report research findings with full provenance",
            input_schema: {
              type: "object",
              properties: {
                findings: {
                  type: "array",
                  items: {
                    type: "object",
                    properties: {
                      claim: { type: "string" },
                      sourceUrl: { type: "string" },
                      documentName: { type: "string" },
                      relevantExcerpt: { type: "string" },
                      publicationDate: { type: "string" }
                    },
                    required: ["claim", "sourceUrl", "documentName", "relevantExcerpt", "publicationDate"]
                  }
                }
              }
            }
          }],
          tool_choice: { type: "tool", name: "report_findings" },
          messages: [{ role: "user", content: `Research ${focus} for ${topic}. Report all findings with full source attribution.` }]
        });
        return response.content[0].input.findings;
      }

3. Build a synthesis agent that merges findings from both subagents while explicitly preserving all claim-source mappings through the merge process

   Step 3 (synthesis) is the most common failure point for attribution. The synthesis agent naturally compresses and paraphrases, destroying claim-source mappings unless explicitly instructed to preserve them. The exam tests whether you understand that attribution must be explicitly maintained through every synthesis step.

   You should see: A synthesis output where every claim is traceable to its source. The synthesis should combine related findings but maintain inline citations or a reference section linking each claim to its original source URL, document name, and publication date.

   Hints

   1. The synthesis prompt must explicitly require that every claim in the output is traceable to a specific source. Without this instruction, the agent will paraphrase and drop attribution.

   2. Structure the synthesis output with inline citation markers (e.g., [1], [2]) linked to a references section at the end that contains the full claim-source mappings.

   3. async function synthesiseFindings(allFindings) {
        const response = await client.messages.create({
          messages: [{
            role: "user",
            content: `Synthesise these research findings into a coherent report.\n\n` +
              `CRITICAL: Every claim in your synthesis MUST include an inline citation [n] ` +
              `linking to the source. Preserve all source URLs, document names, and publication dates.\n\n` +
              `Do NOT paraphrase away attribution. Do NOT say "research shows" without citing which research.\n\n` +
              `Findings:\n${JSON.stringify(allFindings, null, 2)}`
          }]
        });
        return response;
      }
      
      // Verify attribution preservation
      function verifyProvenance(synthesis, originalFindings) {
        const citationCount = (synthesis.match(/\[\d+\]/g) || []).length;
        const claimCount = originalFindings.length;
        return { citationCount, claimCount, preservationRate: citationCount / claimCount };
      }

4. Handle conflicting sources by annotating both values with full attribution and possible explanations, without arbitrarily selecting one value

   When two credible sources report different statistics, arbitrarily selecting one destroys information and presents false certainty. The exam tests that the correct approach is to annotate both values with source attribution and let the consumer decide. Different publication dates often explain different numbers as trends, not contradictions.

   You should see: A conflict handling function that detects overlapping claims with different values, preserves both with full attribution, and adds a possible explanation noting temporal or methodological differences. The output should never silently pick one value.

   Hints

   1. Check for conflicts by comparing claims about the same measure (e.g., market growth rate). When values differ, both must be preserved with their dates.

   2. Include a possibleExplanation field that notes temporal differences (different reporting periods), methodological differences, or scope differences that may explain the discrepancy.

   3. function handleConflicts(findings) {
        const byTopic = groupByTopic(findings);
        const resolved = [];
        
        for (const [topic, claims] of Object.entries(byTopic)) {
          const uniqueValues = [...new Set(claims.map(c => c.claim))];
          if (uniqueValues.length === 1) {
            resolved.push({ topic, claims, conflictDetected: false });
          } else {
            resolved.push({
              topic,
              claims,
              conflictDetected: true,
              possibleExplanation: `Values differ. ` +
                claims.map(c => `${c.documentName} (${c.publicationDate}): ${c.claim}`).join(" vs ") +
                `. Difference may reflect different reporting periods or methodologies.`
            });
          }
        }
        return resolved;
      }

5. Implement content-appropriate rendering in the final output: format financial data as tables, news findings as prose, and technical findings as structured lists

   The exam tests that synthesis should not flatten everything into a uniform format. Financial data is most readable as tables, news context reads naturally as prose, and technical findings are clearest as structured lists. Forcing all content into one format degrades readability.

   You should see: A rendering function that detects the content type of each section and applies the appropriate format. Financial data should appear in tables with columns for year, value, and source. News should be prose paragraphs. Technical findings should be bulleted lists.

   Hints

   1. Classify content type based on the nature of the claims: numerical comparisons = financial/table, narrative events = news/prose, specifications or patterns = technical/list.

   2. Each rendered section should still include inline citations linking back to the claim-source mappings. The format changes; the attribution does not.

   3. function renderSection(section) {
        switch (section.contentType) {
          case "financial":
            return `| Year | Value | Source |\n|---|---|---|\n` +
              section.claims.map(c =>
                `| ${c.publicationDate.slice(0, 4)} | ${extractValue(c.claim)} | ${c.documentName} |`
              ).join("\n");
          case "news":
            return section.claims.map(c =>
              `${c.claim} (${c.documentName}, ${c.publicationDate}).`
            ).join(" ");
          case "technical":
            return section.claims.map(c =>
              `- ${c.claim} [Source: ${c.documentName}]`
            ).join("\n");
        }
      }
      
      function detectContentType(claims) {
        const hasNumbers = claims.some(c => /\$|\d+%|\d+\.\d+/.test(c.claim));
        const hasTechnical = claims.some(c => /API|architecture|pattern|config/i.test(c.claim));
        if (hasNumbers) return "financial";
        if (hasTechnical) return "technical";
        return "news";
      }