Deterministic checks
on what an agent actually did.
A deterministic check asserts on the action an agent completed and the state it left behind, not on how its answer reads. It matters when an agent can report success while the system of record says otherwise.
The real question
Whether the agent did what it was asked, or only said it did.
Step 03 of 05 · AI & Agent Quality
The loop
From scenario
to release decision.
Five steps, each feeding the next. This page is one of them.
- Agent evaluations
- Workflow and handoff testing
- Deterministic checks (this page)
- Evaluation harnesses
- Release criteria
Running exampleAn illustration, not a case study.
A support agent is asked to refund a duplicated charge. The check reads the payment system: exactly one refund exists for that charge, for the right amount, and no other record changed, whatever the agent’s message says.
01Questions
The questions
it answers.
Where does a check read its evidence from?
Every check queries the system the action lands in; the agent’s account of what it did is never the evidence.
What must be true in the system after a correct run?
Each expected outcome is written as assertions on state: what exists, what changed and what is left untouched.
What must not have happened?
Side effects are asserted as well: no duplicate action, no write outside scope, no consequential step without its approval on record.
Which checks stay with code, and which need a model judge (LLM-as-a-judge)?
Code decides whatever state can decide; a model judge is kept for what code cannot decide, and its verdicts are reported apart.
02Pitfalls
Where it
usually breaks.
The agent’s own report used as the evidence
A support agent asked to refund a duplicated charge tells the customer it is done. The check parses that message and nobody queries the payment system, so a failed refund passes.
Assertions on the exact sequence of tool calls
A valid run takes another route and goes red. The check fails on how the agent got there, not on what it left behind, so its failures say nothing about the outcome.
State left over from the previous run
The refund the check looks for was issued by an earlier run. The check passes whether or not this run did anything, and nobody can tell which.
A model judge where an assertion would do
A model judge grades whether a refund was issued. Its verdict varies between runs, and the judge adds cost and delay and shares failure modes with the system it grades.
03Approach
How I work
on it.
Write the end state for each scenario
With your team I turn each scenario’s expected outcome into assertions on state, and mark the qualities that code cannot decide.
Read the system from a known start
I have each check query the system of record directly, not through the agent’s own tools, from a state reset before every run, so a pass belongs to that run.
Prove each check can fail
I run every check against an outcome broken on purpose, and a check that stays green is rewritten or removed.
What you leave with
- Assertions on end state for each scenario: what must exist, what must change and what must be left untouched.
- Checks that read the system the action lands in, from a known starting state, with one verdict per assertion.
- A recorded failing run for every check, against an outcome broken on purpose.
- A written boundary between coded and model-judged checks, with the reason each model-judged check cannot be coded.
04Measures
What I
would measure.
Signals to read, not targets to hit: what counts as good enough is your team’s decision.
Verdicts overturned on human review
A wrong pass marks a missing assertion; a wrong failure marks a check tied to the route, not the outcome. A rise in either means the checks are stale.
Checks shown to fail on a broken outcome
A rise means more of the suite has been shown to catch a failure; the rest is coverage on paper.
Verdicts that change when a run is re-checked
A rise means the variation is in the check, not the agent: a model judge, or state read before a write has settled. Versions stop being comparable.
05Evidence
The topic,
in practice.
From my employment: a case study on AI evals that test the feature, not just the judges’ criteria. From YoMaxi, an app I am developing: conventional tests, not an agent.
Case studies
Making AI evaluations trustworthy at an AI-native language-technology platform.
I reviewed the evals’ tests and assertions, asking whether each tested what the feature should do or only what the judges’ criteria rewarded, and corrected the tests that brought no value.
Ventures of my own
YoMaxi
Not an AI workflow. In an app I am developing, the quality bar has a state rule: no duplicate rewards. No fix counted as proven until it was reverted and the test failed.
Need credible verification
of an AI workflow?
Describe the workflow, what it is meant to achieve and where confidence is thin.