Evaluation, Testing & Optimisation · Lesson 1 of 6

3.1 - Defining Evaluation Metrics: Accuracy, Latency, Cost, Safety and Security

Choose metrics for each quality dimension of a Claude system, decide which are code-graded, model-graded or human-graded, and set thresholds that trace back to explicit success criteria.

An evaluation metric is a decision in disguise. Before you write a single test case you are deciding what “working” means, how you will observe it, and what number is good enough to ship. At the Professional level the question is not what accuracy is, but whether you can name the dimensions that matter for a use case, pick a measurement method for each, and defend a threshold a stakeholder could sign off on. Lesson 3.2 covers the datasets these metrics run on and Lesson 3.3 covers using them to compare versions.

Anthropic’s guidance starts from success criteria, not from tooling. Good criteria are specific (“accurate sentiment classification” rather than “good performance”), measurable (quantitative metrics or well-defined qualitative scales), achievable (grounded in prior experiments and not beyond what current frontier models can do) and relevant to the application’s purpose. The same docs stress that most use cases need multidimensional evaluation: a single number will hide the trade-offs you actually have to manage.

The five metric families

Treat the five families as rows in a metric specification; each needs its own method and threshold.

Accuracy (task fidelity). How well the output does the job, including rare inputs. Anthropic’s docs distinguish task-specific metrics (F1, ROUGE-L) from generic ones (accuracy, precision, recall). Pick the metric whose failure mode matches the business failure mode: a triage classifier that must never miss a fraud report cares about recall on that class, not overall accuracy.
Latency. Distinguish time to first token (what a streaming user feels) from total response time (what a pipeline feels). Report percentiles, not means; the docs’ own example criterion is a percentile (95% of responses under a stated time).
Cost. Measure cost per completed task, built from the token counts in each response’s usage object plus retries and escalations. Cost per token rewards a cheap configuration that fails often.
Safety. Harmful or policy-violating output, and also the opposite failure of refusing legitimate requests. Even “hazy” goals can be quantified, for example a maximum share of outputs flagged for toxicity across a stated number of trials.
Security. Resistance to jailbreaks and to direct and indirect prompt injection, and whether a successful injection can reach sensitive data or actions. Anthropic recommends red-teaming your own workflow with documents, emails and tool outputs that deliberately contain injection attempts, which yields a measurable attack success rate. Guardrail design is Lesson 4.1.

Key concept: a metric is four things, not one

A usable metric specifies what is measured (a behaviour tied to a success criterion), how it is graded (code, model or human), a threshold (what counts as good enough) and a consequence (block the release, page someone, or just track it). If any of the four is missing, the metric will be argued about after the fact instead of deciding anything.

Choosing the grader: code, model or human

Anthropic’s eval guidance is to automate wherever you can and to structure the task so automated grading is possible, and that volume beats hand-graded quality: more questions with slightly noisier automated grading are better than a few beautifully hand-graded ones. Its agent-evals write-up frames the three grader types by their trade-offs.

Grader	Strengths	Weaknesses	Use for
Code-graded (exact match, regex, schema or state checks)	Fast, cheap, objective, reproducible, easy to debug	Brittle to valid variation; no nuance	Labels, extracted fields, format validity, latency, cost, final state
Model-graded (LLM-as-judge, Likert, binary or ordinal rubric)	Flexible, scales, handles open-ended output	Non-deterministic, costs tokens, needs calibration	Tone, empathy, faithfulness to a source, whether instructions were followed
Human	Gold-standard judgement	Slow, expensive, needs expert access	Calibrating the judge, new failure modes, high-stakes samples

The docs list concrete methods: exact match for categorical answers, embedding cosine similarity for consistency across paraphrased inputs, ROUGE-L for overlap with a reference summary, and LLM-based Likert, binary and ordinal scales for qualities such as empathy or context use. Three practices make model grading trustworthy: use a different model as judge than the one that produced the output; write a structured rubric and grade each dimension with its own isolated judge call; and give the judge a way out (return “unknown” when evidence is insufficient), then calibrate against human graders on a sample.

Also grade what the system produced, not the path it took: agents find valid routes the eval author did not anticipate, so rigid step checks penalise correct behaviour. Where an outcome is partly right, build in partial credit.

Common exam distractor

Two answers look sophisticated and are wrong. The first is a public benchmark score or a single headline accuracy figure offered as evidence that your system is fit for your task; generic benchmarks do not mirror your task distribution or your edge cases. The second is an LLM judge from the same model family as the generator, with no fixed rubric and no human calibration. The correct direction is task-specific evals, a fixed rubric, a different judge model, and a human-labelled sample to check the judge.

Thresholds tied to success criteria

A threshold is legitimate only if it traces to something a stakeholder cares about. Anthropic’s docs model this with a multidimensional criterion for a sentiment classifier: an F1 floor on a held-out set, a minimum share of non-toxic outputs, a share of errors that must be low-severity, and a percentile latency bound. Treat the figures as an illustration of the shape of a criterion, not as targets. Note the third item: it weights errors by severity, and admits that “inconvenience” and “egregious” must be defined. A 95% score is meaningless if the failing 5% are the ones that trigger a refund or a compliance report.

Separate gate metrics (must pass to release), guardrail metrics (must not regress while you optimise something else, Lesson 3.3) and diagnostic metrics (debugging aids that never block).
Set thresholds from consequences: what does one false negative cost compared with one false positive?
For agents, distinguish pass@k (succeeds at least once in k tries) from pass^k (succeeds on every try). An agent that must behave reliably on every conversation should be held to the stricter second reading.
Validate the metric itself: read transcripts to confirm low scores reflect real failures, since ambiguous task specs or grader bugs can depress scores for reasons unrelated to model capability.
Tie cost and latency to the SLO you promised (Lessons 1.3 and 2.6).

Worked example: a contract-clause extraction assistant

Claude extracts termination dates and liability caps from vendor contracts and drafts a two-sentence risk note. Accuracy: code-graded exact match on normalised dates and amounts against attorney labels, per field, with recall on liability caps as the gate because a missed cap is costly. Faithfulness of the note: model-graded binary rubric (every claim supported by the clause text, “unknown” allowed), calibrated on a human-reviewed sample. Latency: a percentile of end-to-end time. Cost: tokens per contract including retries. Security: the rate at which an instruction injected into a contract PDF changes the output. Each metric maps to a stated criterion and states the consequence of a miss.

Exam traps

Using a public benchmark score or a generic accuracy number as the evidence that a system meets the business requirement.

Anthropic’s guidance is to design evals that mirror your real task distribution, including edge cases. A generic score says nothing about your data, your failure costs, or your latency and cost constraints.
Optimising and reporting one headline metric (for example overall accuracy) and treating latency, cost, safety and security as afterthoughts.

Most use cases need multidimensional evaluation. A configuration that improves accuracy while doubling p95 latency or weakening injection resistance may be unshippable, and a single number hides that trade-off.
Choosing an LLM judge for every metric, or letting a model grade output from its own family with no fixed rubric and no human check.

Where a task has one correct answer, code grading is cheaper, deterministic and unambiguous. Where a judge is needed, use a different model, a structured per-dimension rubric with an unknown option, and calibrate against human graders.
Reporting an average score and treating a high pass rate as proof of quality, without weighting errors by severity or by class.

Aggregate scores can be dominated by easy cases while the rare, costly failures (missed fraud, wrong liability cap, leaked data) go unmeasured. Thresholds should be derived from consequences, with per-class or severity-weighted reporting.

Practice question

A team is launching a Claude feature that summarises customer support tickets for agents. Requirements: summaries must not contain claims unsupported by the ticket, the tone must be neutral, 95% of summaries must arrive within a stated time, and monthly spend must stay within budget. The team proposes to evaluate with one ROUGE-L score against reference summaries and ship when it exceeds a chosen number. What is the best improvement?

A Keep ROUGE-L as the single metric but raise the threshold, increase the number of reference summaries and have two agents write each reference so that the score is more reliable and harder to game.
More references and a higher bar do not fix the mismatch. ROUGE-L measures overlap with a reference; it does not measure unsupported claims, tone, latency or cost, so the requirements would still be unmeasured.
B Replace ROUGE-L with a single LLM-judge quality score from the same model that writes the summaries, using a detailed prompt that covers faithfulness, tone and concision in one overall rating.
This still collapses several dimensions into one number and adds self-preference risk from using the generator’s own model family as the judge, with no calibration.
C Define a metric per requirement: a calibrated model-graded faithfulness rubric with an unknown option, a tone rubric, a latency percentile from recorded runs and cost per summary from usage data, each with a threshold. Correct
Each requirement gets a method suited to it (model-graded where judgement is needed, code-graded for latency and cost), with thresholds traceable to the success criteria and a human-calibrated judge.
D Skip automated metrics and have the support team read a handful of summaries each week to decide whether the feature is good, escalating anything that looks wrong to the product owner.
Manual spot checks are slow, unrepresentative and unrepeatable. Human review is valuable for calibration and new failure modes, but volume with automated grading is the primary approach.

Build exercise: Write a metric specification and grade it: code, model and latency metrics for one use case

Intermediate · 60 minutes

You'll practice:

Turn a use case into a metric specification with a method, owner, threshold and consequence for each dimension
Implement a code-graded metric and a model-graded rubric metric with a fixed, structured judge prompt
Compute percentile latency and cost per completed task from recorded runs rather than from averages
Calibrate the judge against a small human-labelled sample and record the disagreement

Pick a real or realistic Claude use case. Write a one-page metric specification as a table with one row per dimension (accuracy, latency, cost, safety, security). For each row record: the success criterion it traces to, the measurement method (code, model or human), the threshold, and the consequence of missing it (gate, alert or track).

The exam rewards metrics that trace to success criteria. Writing the table first forces you to justify each threshold before you have data that might tempt you to fit the threshold to the result.

You should see: A five-row specification where no threshold is stated without a criterion, every row has a grader type, and at least one row is explicitly marked as a release gate and one as a guardrail.

Hints

For each row, ask who would be unhappy if this metric failed, and what they would say the acceptable level is.

Start from consequences: which error is most expensive (missed item, wrong item, leaked item, slow response)? Choose the metric and its direction (recall, precision, percentile) accordingly.

Example row: Dimension: accuracy on liability caps. Criterion: a missed cap must be rare. Method: code-graded exact match on normalised amount vs attorney labels. Threshold: recall at or above the value the legal owner signs off. Consequence: release gate. Second example row: Dimension: latency. Criterion: agents wait while the summary loads. Method: code-graded percentile from recorded runs. Threshold: p95 under the agreed bound. Consequence: alert, and gate if the model or effort setting changes.

Write 10 to 15 test cases as JSON. Include at least two cases with an unambiguous expected answer, two edge cases (empty or irrelevant input, very long input) and one adversarial case that embeds an instruction inside the data. Implement the code-graded metric (exact match on normalised values) for the cases that have an expected answer.

Code grading is the cheapest and most reliable method when a task has a single correct answer, and the exam expects you to reach for it before a model judge.

You should see: A JSON file of cases and a function that returns pass or fail per case plus a pass rate reported separately for the exact-match subset, the edge cases and the adversarial case.

Hints

Which of your fields can be reduced to a normalised string or number so that two correct answers always compare equal?

Normalise before comparing (strip, lowercase, parse dates and amounts to canonical form) and keep the comparison strict otherwise, so partial matches are not silently accepted.

import json, re

def norm(v):
    return re.sub(r"\s+", " ", str(v)).strip().lower()

def grade_exact(output: dict, expected: dict) -> dict:
    return {k: norm(output.get(k)) == norm(v) for k, v in expected.items()}

cases = json.load(open("cases.json"))
scored = [(c["id"], grade_exact(run_system(c["input"]), c["expected"])) for c in cases if "expected" in c]
for cid, fields in scored:
    print(cid, fields)

Implement a model-graded faithfulness metric. Send the source, the system output and a fixed rubric to a judge model that is different from the model under test, and require a structured verdict of pass, fail or unknown plus a one-sentence reason. Run it over every case that has free-text output.

Open-ended output cannot be exact-matched. A fixed, structured rubric with an unknown option and a separate judge model addresses the two main weaknesses of LLM grading: drift in criteria and self-preference.

You should see: A function that returns a parsed verdict for every free-text case, with any unknown verdicts counted and reported separately rather than silently treated as pass.

Hints

What could you put in the judge prompt so that the answer is trivially machine-parseable and cannot be confused with prose?

Constrain the output with a JSON schema through the output_config format option, keep the rubric text constant across runs, and validate the verdict value in your code.

import json, anthropic
client = anthropic.Anthropic()
JUDGE_MODEL = "claude-opus-5"   # must differ from the model under test; check the models page for current IDs
SCHEMA = {"type": "object",
          "properties": {"verdict": {"type": "string"}, "reason": {"type": "string"}},
          "required": ["verdict", "reason"], "additionalProperties": False}
RUBRIC = ("Return verdict 'pass' only if every factual claim in the summary is supported by the source. "
          "Return 'fail' if any claim is unsupported. Return 'unknown' if the source is insufficient to decide.")

def judge(source: str, summary: str) -> dict:
    r = client.messages.create(
        model=JUDGE_MODEL, max_tokens=300,
        output_config={"format": {"type": "json_schema", "schema": SCHEMA}},
        messages=[{"role": "user",
                   "content": f"<source>{source}</source>\n<summary>{summary}</summary>\n{RUBRIC}"}])
    out = json.loads(next(b.text for b in r.content if b.type == "text"))
    assert out["verdict"] in {"pass", "fail", "unknown"}
    return out

Record latency and token usage for every run of your system (time each call, and store input, output and cache token counts from the response usage object). Compute p50 and p95 latency, and cost per completed task using the current per-token rates from Anthropic’s pricing page, counting retried calls against the task they belong to.

Percentiles show the tail that users and SLAs feel, and cost per completed task exposes retry and escalation overhead that a per-request average hides.

You should see: A small report with p50 and p95 latency, tokens per task, and a cost-per-completed-task figure computed from rates you looked up in the docs and did not memorise.

Hints

If one task needed two calls because the first failed validation, how many tasks did you complete and how many calls did you pay for?

Log one record per call with a task_id, then group by task_id: sum tokens across the group, time the whole group, and count the task once.

import math, time

def pct(vals, p):
    s = sorted(vals)
    return s[max(0, math.ceil(p / 100 * len(s)) - 1)]

records = []   # one dict per call: task_id, seconds, input_tokens, output_tokens, cache_read, cache_write
def timed_call(task_id, **kw):
    t0 = time.perf_counter()
    r = client.messages.create(**kw)
    u = r.usage
    records.append({"task_id": task_id, "seconds": time.perf_counter() - t0,
                    "input_tokens": u.input_tokens, "output_tokens": u.output_tokens,
                    "cache_read": u.cache_read_input_tokens or 0,
                    "cache_write": u.cache_creation_input_tokens or 0})
    return r

by_task = {}
for rec in records:
    by_task.setdefault(rec["task_id"], []).append(rec)
task_secs = [sum(x["seconds"] for x in g) for g in by_task.values()]
print("p50", pct(task_secs, 50), "p95", pct(task_secs, 95))

Hand-label 6 to 8 of the judged cases yourself, blind to the judge’s verdict. Compare your labels to the judge, list every disagreement, and decide for each whether the fix belongs in the rubric wording, the judge choice or the dataset. Finish by marking which metrics in your specification are gates and which are guardrails.

Calibration is what makes a model-graded metric defensible. The disagreements are the evidence that tells you whether the metric can be trusted as a gate.

You should see: A short table (case id, judge verdict, your verdict, agree or disagree, proposed fix) and an updated specification with gate and guardrail labels.
Hints
1. ```
If the judge is more lenient than you on unsupported claims, is the problem the rubric text, or the model’s tendency?
```
2. ```
Tighten the rubric wording first (define what counts as supported), then re-run; only change the judge model if disagreement persists after the rubric is clear.
```
3. ```
Example row: case c07, judge pass, human fail, reason: summary states a delivery date that appears only in a different ticket. Fix: rubric wording now says that a claim must be supported by the text inside the source tags, not by general plausibility. Re-run c07 and the other cases that share that pattern, and record the new agreement count.
```

Sources

Define your success criteria (Claude docs) - Anthropic
Create strong empirical evaluations (Claude docs) - Anthropic
Demystifying evals for AI agents (Anthropic Engineering) - Anthropic
Mitigate jailbreaks and prompt injections (Claude docs) - Anthropic
Reducing latency (Claude docs) - Anthropic