Agent evaluations
built from real use and known failures.
An agent evaluation, or agent eval, starts from scenarios built from real use and known failures, each a request, a starting state and an expected outcome. It matters when confidence in your agent rests on spot checks of outputs that read well.
The real question
Whether success was written down before the run, or recognised in the output after it.
Step 01 of 05 · AI & Agent Quality
The loop
From scenario
to release decision.
Five steps, each feeding the next. This page is one of them.
- Agent evaluations (this page)
- Workflow and handoff testing
- Deterministic checks
- Evaluation harnesses
- Release criteria
Running exampleAn illustration, not a case study.
A support agent is asked to refund a duplicated charge. The scenario states the request, the account’s context and the outcome that counts as success: one refund for the duplicated charge, the customer told, nothing else changed.
01Questions
The questions
it answers.
What counts as success for each request?
Each scenario states the request, its context, the starting state and the outcome that counts as success, in terms a check can verify.
Where do the scenarios come from?
The set is built from the kinds of requests the workflow actually receives and from failure reports that already exist.
Which cases does the set not reach?
Coverage is reviewed against what the agent can do, not only what it was asked; cases real use has not produced are added and marked as constructed.
When does a passing set stop telling you anything?
New failures enter the set as scenarios, and scenarios every version passes are replaced or kept on purpose as regression guards.
02Pitfalls
Where it
usually breaks.
Expected outcomes stored as reference answers
Runs are graded by closeness to a sample text. A correct result worded differently fails, a fluent wrong one passes, and the score follows phrasing, not the task.
A set written only by the team that built the agent
The set repeats the assumptions already in the prompt, so the agent passes. Real requests look different, and those failures are found in use, not in the run.
A scenario set nobody refreshes or retires
Prompts are tuned against the same scenarios until every one passes, while the product and its use move on. The pass rate climbs and tells you less with each release.
No scenario where the right outcome is to stop
Every scenario expects a completed action. Nothing shows whether the agent refuses, asks or waits for approval when it should, so over-reach never fails a run.
03Approach
How I work
on it.
Collect real requests and known failures
With your team I gather what the workflow is really asked to do and the failures already reported, ranked and grouped by frequency and by what a failure costs.
Write the outcome before the run
With your product and domain experts I write each scenario as request, starting state and expected outcome, including cases where the agent should not act.
Review coverage and agree the upkeep
I review the set against the cases that matter most, record what it does not reach, and agree how failures become scenarios and when one is retired.
What you leave with
- Scenarios for critical journeys, failure paths and recovery, each with its request, starting state and expected outcome.
- A record of where each scenario came from: a kind of real request, a reported failure or a risk named in review.
- A coverage review against the cases that matter most, with what the set does not reach written down.
- An upkeep rule: how a failure becomes a scenario, which scenarios stay out of prompt tuning and when one is retired.
04Measures
What I
would measure.
Signals to read, not targets to hit: what counts as good enough is your team’s decision.
Reported failures with no matching scenario
A rising share means the set has drifted from how the workflow is used; each such failure is a scenario the set was missing.
Scenarios that pass on every version
A growing share means the set can no longer tell one version from the next, and needs newer or harder cases.
Reviewer disagreement on expected outcomes
A rise means outcomes are underspecified; a check written from a disputed outcome passes or fails on one reviewer’s reading.
05Evidence
The topic,
in practice.
From my employment: a case study of AI evaluations reviewed to make sure they test the feature, with the limits of its evidence stated.
Case studies
Making AI evaluations trustworthy at an AI-native language-technology platform.
The evals were reviewed to make sure they tested the feature, not just the judges’ criteria, and dashboards on live customer data showed where success rates were low, so coverage could be added there.
Need credible verification
of an AI workflow?
Describe the workflow, what it is meant to achieve and where confidence is thin.