Integration · Lesson 6 of 8

1.6 - Retrieval Strategies Matched to Data Shape and Query Pattern

Choose among semantic, keyword, hybrid, reranked, structured and agentic retrieval by matching the strategy to the data shape and the query pattern, and evaluate each stage separately.

There is no best retrieval method, only one that fits the shape of the data (unstructured prose, semi-structured documents, tables, code or logs) and the pattern of the query (a fuzzy concept, an exact identifier, an aggregate over rows, a multi-hop question, a conversational follow-up). Scenario questions in this area describe a data shape and a failing query type and ask you to pick the method that fits, or to spot the method that cannot possibly work. Chunking and indexing are Lesson 1.5; this lesson is about choosing how to search once the index exists.

The toolbox and where each tool breaks

Strategy	Strength	Typical failure
Semantic (dense embeddings)	Paraphrases and concepts	Exact identifiers, rare tokens, part numbers, sometimes negation and numbers
Keyword (BM25)	Exact terms: error codes, names, SKUs	Synonyms and paraphrase; the user's words differ from the document's
Hybrid (both, merged by rank fusion)	Covers concept and exact-match queries	Two indexes to run; fusion and weights to tune
Reranking	Re-scores a wide candidate set against the query for better top-k precision	Cannot surface documents that were never in the candidate set; adds a model call
Metadata filtering	Narrows by tenant, date, type or entitlement	Only as good as the metadata; wrong filters silently hide the answer
Structured or SQL retrieval	Filters, joins and aggregates over tables	Wrong or unsafe queries; needs schema context and safeguards
Query rewriting or decomposition	Turns follow-ups into standalone queries; splits multi-part questions	Extra call; the rewrite can drift from the user's intent
Agentic retrieval	The model searches iteratively and follows references	More turns, tokens and latency; depends on good search tools

Anthropic's Contextual Retrieval experiments illustrate the layering: combining contextual embeddings with contextual BM25 through rank fusion cut the top-20 retrieval failure rate further than embeddings alone (5.7% to 3.7% to 2.9%), and adding a reranker over a wider candidate set (150 narrowed to 20) reached 1.9%. Those are numbers from their datasets, and the lesson is the pattern, not the digits: each stage fixes a different failure, and each adds cost.

Matching strategy to data shape and query pattern

Data and query	Fit	Why
Prose knowledge base, conceptual questions	Dense retrieval, plus a reranker if top-k precision is weak	Meaning matters more than exact wording
Prose that queries reference by code, ID or product name	Hybrid	Dense vectors are unreliable on rare exact tokens; BM25 is strong there
Orders, metrics or events in tables; "how many", "total", "top 5"	Structured query (parameterised SQL or a fixed set of query tools)	Similarity search cannot count or sum; embedding a table does not make it computable
Large, fast-changing codebase or log store; exploratory questions	Agentic search (grep and glob style tools), possibly with an index	No index staleness; the model can follow references
Multi-hop questions across documents	Query decomposition or agentic retrieval	The second lookup depends on the first result
Chat follow-ups such as "and for contractors?"	Rewrite with history into a standalone query	The raw follow-up has no searchable content
Multi-tenant or permissioned data	Mandatory metadata pre-filter	Access must be enforced before content reaches the model

Key concept: match the method to the shape of the answer

Ask what the answer looks like. A passage that means the same thing as the question wants semantic search. A row that contains a specific token wants keyword search. A number computed over many rows wants a query engine. A chain of dependent lookups wants an agent or a decomposition. Popularity of a technique is not evidence.

Agentic retrieval and structured data

Agentic retrieval lets Claude decide what to search for, inspect results, and search again. Anthropic's context engineering write-up describes the hybrid used by Claude Code: some context (project instructions) loaded up front, and the rest explored just in time with tools such as grep and glob, keeping lightweight identifiers rather than pre-loading content. It states the cost plainly: runtime exploration is slower than retrieving pre-computed data, and it needs careful tool design. Tool results can carry search_result blocks so the model's citations point at your sources (Lesson 1.5).

Structured retrieval needs different safeguards. Exposing free-form SQL to the model widens the blast radius (Lesson 1.1), so prefer parameterised query tools or a constrained query builder, run them on a read-only credential, expose the schema as context, cap result sizes and validate generated queries. Aggregates should come from the database, not from the model summing rows it was shown.

Cost, evaluation and safety

Each stage adds latency. A reranker is a model call, query rewriting is a model call, and each agentic search is a turn. Treat depth (how many candidates, how many searches) as a swept variable, as in Lesson 1.3.
Evaluate by query type. Overall recall hides that exact-ID queries fail while concept queries succeed. Tag each test query with its pattern, and measure candidate recall (is the gold item in the wide set?) separately from top-k precision (is it near the top?). Recall problems need a different retriever or query; precision problems need reranking.
Retrieved text is untrusted input. A document can contain instructions aimed at the model. Enforce access control by filtering at retrieval time using the user's entitlements, not by asking the model to ignore documents it should not see (Lesson 1.2), and log queries and results without their content by default (Lesson 1.4).

Common exam distractor

"Use vector search for everything" fails on identifiers and aggregates. "Add a reranker" is the wrong fix when the gold document is not in the candidate set, because a reranker only reorders what was retrieved. "Embed the whole orders table so the model can answer totals" asks similarity to do arithmetic. And "always retrieve more chunks" trades tokens and latency for a recall gain you should have measured first.

Exam traps

Using pure semantic search for a corpus where users query by error code, SKU or exact name

Dense embeddings capture meaning and can miss rare exact tokens. Keyword (BM25) matching handles them well, so a hybrid that merges both rankings is the usual fix.
Embedding a structured orders or metrics table so the model can answer totals and top-N questions

Similarity search returns rows that look related; it cannot count, sum or rank across the whole table. Aggregate and filter questions belong in a structured query (parameterised SQL or fixed query tools) on a read-only credential.
Adding a reranker to fix a system where the correct document is not retrieved at all

A reranker reorders a candidate set; it cannot add documents that are missing from it. Diagnose candidate recall first, then fix the retriever, chunking or query (for example with hybrid search or rewriting).
Enforcing document permissions by telling the model which documents the user may see

Instructions are not enforcement and retrieved text can contain injected instructions. Filter by the user's entitlements at retrieval time so unauthorised content never reaches the model.

Practice question

A support knowledge base holds product manuals and troubleshooting articles. Users ask two kinds of questions: conceptual ones ('how do I speed up syncing?') and lookups by identifier ('what does ERR_4471 mean on device SKU-77821?'). The current dense-embedding retriever handles the conceptual questions well but often misses the identifier lookups even though the exact codes appear in the articles. Which change best fits the failure?

A Switch to an embedding model with more dimensions, re-embed the whole knowledge base, and re-run the identifier queries to see whether the exact codes now rank higher
More dimensions can help general similarity but do not turn rare exact tokens into reliable matches. The failure is about exact-term matching, which a keyword index handles directly.
B Add a BM25 keyword index beside the dense index, merge the two rankings with rank fusion, and check recall separately for identifier and conceptual queries Correct
BM25 matches the exact codes that dense vectors miss, dense retrieval keeps the conceptual coverage, and fusing the rankings serves both query types. Slicing the evaluation by query type verifies the fix helps the failing slice without hurting the other.
C Add a reranker on top of the existing dense retriever's top 50 candidates so that the articles containing the identifier are moved to the top of the list
If the article with the code never appears in the dense top 50, the reranker never sees it. Reranking improves ordering of candidates, not candidate recall.
D Have Claude answer identifier questions from its own knowledge when retrieval returns nothing relevant, and flag those answers as unverified
Device-specific codes and SKUs are your data. Answering from model memory risks confident fabrication, which is worse than a miss.

Build exercise: Run a retrieval bake-off across query patterns

Advanced · 120 minutes

You'll practice:

Classify queries by pattern and data shape and predict which retrieval strategy fits
Measure candidate recall and top-k precision separately, sliced by query type
Implement rank fusion and compare hybrid against single-method baselines
Decide when a reranker helps and when it cannot
Route aggregate questions to a safe structured-query path

Assemble a test set of at least 30 queries over a corpus that mixes prose and at least one table. Tag each query with a pattern: concept, exact identifier, aggregate, multi-hop or follow-up, and record the gold result for each.

A bake-off is only meaningful if it reflects the real mix of queries and lets you slice results by pattern.

You should see: A spreadsheet with query, pattern tag, gold document or gold answer, and (for follow-ups) the preceding turn.
Hints
1. ```
Which patterns do you expect each strategy to fail on before you run anything?
```
2. ```
Write your predictions in a column first. Aim for roughly six queries per pattern so slices are not empty.
```
3. ```
Example rows: 'how do I speed up syncing' | concept | doc 12. 'ERR_4471 SKU-77821' | identifier | doc 88. 'total refunds by region last quarter' | aggregate | SQL result. 'and for contractors?' | follow-up | doc 31 with prior turn.
```
Implement dense-only and BM25-only baselines. For each query record whether the gold item appears in the top 5 and top 20, and report recall by pattern tag.

Two baselines show each method's blind spots and give the fusion step something to beat.

You should see: A table of recall@5 and recall@20 per pattern per method, with identifier queries likely favouring BM25 and concept queries favouring dense.
Hints
1. ```
Where do the two methods disagree, and does one ever find what the other misses?
```
2. ```
Keep the chunking identical across methods so only the retriever varies. Compute recall from the ranked lists you already have.
```
3. ```
For each method, ranks = search(query, k=20); hit5 = gold in ranks[:5]; hit20 = gold in ranks[:20]. Group by pattern tag with a dictionary of lists and average the booleans.
```

Implement reciprocal rank fusion over the two ranked lists and evaluate the hybrid the same way.

Rank fusion is a simple, tunable way to combine methods without comparing their raw scores, which are on different scales.

You should see: A third row per pattern for the hybrid, ideally at least as good as the better baseline on each tag.

Hints

Why is fusing ranks safer than adding raw BM25 and cosine scores?

For each document, sum 1 / (k + rank) over the lists it appears in, where k is a smoothing constant (60 is a common default), then sort by the total.

def rrf(rankings, k=60):
    scores = {}
    for ranking in rankings:
        for rank, doc_id in enumerate(ranking, start=1):
            scores[doc_id] = scores.get(doc_id, 0) + 1 / (k + rank)
    return sorted(scores, key=scores.get, reverse=True)

Add a reranker over the top 50 hybrid candidates and measure precision at 5, latency added and the queries where it changed the outcome. Then deliberately test a query whose gold document is outside the top 50.

This shows both what reranking is for (ordering candidates) and what it cannot do (create candidates).

You should see: A before-and-after precision@5, the added latency per query, and at least one example where reranking could not help because the gold document was missing from the candidates.
Hints
1. ```
For the failed example, at which stage did the gold document get lost?
```
2. ```
Call your reranker with the query and the candidate texts and keep only the top 5. Log the time of the rerank call separately.
```
3. ```
Report three numbers per pattern: recall@50 (candidate recall), precision@5 before rerank, precision@5 after rerank. If recall@50 is low, no reranker will fix it.
```
Handle aggregate queries: design a parameterised, read-only query tool, route aggregate-tagged queries to it, and write the decision matrix that maps each pattern in your test set to a strategy with the trade-off you accepted.

Aggregates are the pattern retrieval methods cannot serve. The matrix is the artifact you would defend in a design review.

You should see: A working tool signature (for example refunds_by_region(quarter)), a read-only credential note, and a completed matrix.
Hints
1. ```
What stops the model from asking the tool for something it should not see?
```
2. ```
Expose named parameters instead of raw SQL, use a read-only database role, add a row limit and log each call with the user identity.
```
3. ```
Matrix rows: concept -> dense + rerank (cost: one extra call). identifier -> hybrid (cost: second index). aggregate -> parameterised SQL tool on read-only credential (cost: tool design). follow-up -> query rewrite with history (cost: one extra call, risk of drift).
```

Sources

Introducing Contextual Retrieval (Anthropic) - Anthropic
Effective context engineering for AI agents (Anthropic Engineering) - Anthropic
Search results content blocks (Claude API docs) - Anthropic
Embeddings (Claude API docs) - Anthropic
Citations (Claude API docs) - Anthropic