Discovery is the phase in which the architecture is actually decided; the design document just writes it down. A stakeholder rarely arrives with a requirement. They arrive with a solution (“we need an agent that approves refunds”, “we want RAG over our wiki”) wrapped around a problem they may not have fully described. Your job in discovery is to separate the two, find the constraints that will eliminate options later, and agree on how success will be measured before you recommend a model, a pattern or a platform.
Scenario questions here usually hinge on sequencing: an option that recommends a model tier, a multi-agent design or a build plan before use case, data, constraints, risk and success metrics are pinned down is almost always the distractor.
Why premature recommendations fail
Three failure modes recur. Anchoring: the first architecture you sketch becomes the one everyone defends, even when later facts contradict it. Solution smuggling: the stated requirement is really an implementation (“use a vector database”), which hides the underlying need and shuts out cheaper options. Capability-first thinking: choosing the most capable model or most autonomous pattern because it impresses, not because the task needs it. Anthropic’s guidance on agents is to find the simplest solution that works and add complexity only when it demonstrably improves outcomes, since agentic systems typically trade latency and cost for task performance. Discovery is where you learn whether that trade is worth it, and sometimes that a deterministic program or one well-prompted call is enough.
A session structure that covers the ground
Organise the conversation around six areas and keep a running list of assumptions and open questions alongside the answers. The table shows what each area is really buying you, that is, which design decision the answers feed.
| Area | Questions that matter | Design decision it feeds |
|---|---|---|
| Use case and baseline | What job is being done today, by whom, how often, at what cost and error rate? What does the user do with the output? | Whether an LLM is needed; the baseline any solution must beat |
| Data | Where does the input live, how clean is it, how fresh must answers be, how sensitive is it, who may see it, how long may it be kept, where may it be processed? | Retrieval design, redaction, platform and data-handling arrangement |
| Constraints | Latency budget, peak volume, budget ceiling, integration points, languages, availability expectations, team skills, deadline | Model tier, sync vs. asynchronous processing, capacity planning |
| Risk | What is the worst plausible wrong output? Who is harmed, how fast, and can it be reversed? What review capacity exists? | Human-in-the-loop placement, verification steps, autonomy level |
| Success metrics | What would make you call this a success in six months? What number, measured how, against what baseline? | Eval design, launch gates, SLOs (see lesson 5.3) |
| Stakeholders and ownership | Who decides, who is affected, who supports it, who is not in the room yet? | Governance, escalation, change control |
Key concept
The output of discovery is not a model choice. It is a short, evidence-backed set of requirements, each traceable to a stakeholder answer or a real sample, plus explicit assumptions and open questions. A model and architecture recommendation is then a function of that set, and you should be able to say which answer would change it.
Turning a wish into a measurable success criterion
“High accuracy” and “better customer experience” are wishes. Anthropic’s documentation on defining success says criteria should be specific, measurable, achievable and relevant, and that most use cases need several criteria at once. Its example categories include task fidelity (with edge cases), consistency, relevance, tone, privacy preservation, context use, latency and price. Ask stakeholders to trade these off explicitly: is a slower, more accurate answer acceptable, and is a wrong answer worse than no answer? A usable criterion looks like “at least X% of tickets in the agreed test set are triaged to the correct queue, measured against senior-agent labels, with no cases in the ‘legal threat’ category misrouted.” The threshold is a business decision; your job is to make it measurable and to ask for real, messy examples so the test set mirrors the actual task distribution, including irrelevant, overlong, ambiguous and adversarial inputs.
Because Claude output is probabilistic, also ask how much variation is tolerable; that seeds the error-budget conversation in lesson 5.3.
Surfacing hidden requirements
Most expensive surprises are requirements nobody stated because they seemed obvious. Techniques that reliably expose them:
- Ask for the worst failure. “Describe the worst thing this system could plausibly do, and what happens next.” The answer defines autonomy limits and review points.
- Ask for 20 real inputs. Real data reveals scanned PDFs, mixed languages, missing fields and internal jargon that a verbal description never will.
- Ask who is not in the room: security, legal, support, downstream owners and the people whose work changes.
- Probe the “obvious” non-functionals: audit trail, explainability to a regulator or customer, retention of prompts and outputs, language coverage, accessibility, on-call ownership, and who maintains the prompts.
Several of these have concrete architectural consequences on Claude. Anthropic documents data-handling arrangements for its API (zero data retention and HIPAA readiness, with a feature-eligibility table), geographic controls for where inference runs, and organisation-level usage tiers, spend limits and rate limits that the docs describe as maximums rather than guaranteed minimums. On partner clouds the cloud provider is the data processor under its own terms. So “where may this data go?” and “what is the peak load?” can rule out a platform, a feature or a launch date. Treat these as architecture-level implications to confirm with legal, security and current vendor documentation; this is not legal advice.
Common exam distractor
Watch for options that answer a vague request by immediately selecting the most capable model, proposing a multi-agent design, or building a demo “to get buy-in”, and for options that defer data-sensitivity and compliance questions until a prototype works. Both skip the questions that decide platform, feature set and acceptance test. Equally weak: asking the sponsor to pick a model tier from a menu. The strong option gathers missing information in a structured way, states assumptions, and defines measurable success first.
Closing discovery
End with a written summary the sponsor can correct: problem and baseline, scope, data inventory and sensitivity, constraints, risk appetite and review assumptions, success criteria with thresholds and measurement, decision owner, and ranked open questions with owners and dates. Include a plain go/no-go statement: sometimes the honest finding is that rules, search or a workflow tool solves the problem, or that the data or review capacity does not yet exist. Discovery is not one-shot; new answers reopen old ones, so version the summary. It later feeds the decision records in lessons 5.2 and 5.4.