AI Agents

How do you evaluate an AI agent, beyond just checking the final answer?

By Interview Clozo · Published 27 October 2026 · Last verified 26 October 2026 · 13 min read

How do you evaluate an AI agent, beyond just checking the final answer? Grade the outcome (the state the agent leaves in its environment), the trajectory (its tool calls) and consistency across repeated trials. An AI agent is a program that runs a language model in a loop with tools, so a message can hide a wrong action. The follow-up is pass^k: at 75% success per trial, an agent passes eight trials in a row about one time in ten.¹

In this article

Before you start
Why interviewers ask this
Agent evaluation from first principles
The Staff-level answer
The rubric
Three variants of this question
Common mistakes
Follow-up questions
Sources

Before you start

This page is one step of the AI agent architecture learning path.

Why interviewers ask this

Evaluating agents is where an agent design stops being a diagram and becomes something a team can ship. A system that calls tools and changes state can fail in ways a chat answer cannot. Interviewers use the question to see whether a candidate separates what an agent says from what it does.

A weak answer is "I check whether the final answer is correct". That describes testing a chatbot. A strong answer names the outcome check, the trajectory and repeated trials. A Staff-level answer also says which grader owns which check, why one exact tool sequence is the wrong reference, and how the pass rate behaves when users need the agent to work every time. Candidates report this question in agent-design and machine learning system design rounds; pair it with agentic AI interview questions for the neighbouring ones.

Agent evaluation from first principles

Outcome check: An outcome check is a grader that reads the state an agent left in its environment, such as a database row or a file, and compares it with the state the task requires, so a confident final message cannot pass a task the agent did not complete.
Trajectory match: A trajectory match is a grader that compares the ordered list of tool calls an agent made with a reference list under a stated rule, such as exact order, any order, no extra tools or at least the reference tools, so it scores the path taken and not only where the agent ended.

Each step catches a failure the final answer hides.CC BY 4.0 — source: How do you evaluate an AI agent, beyond just checking the final answer?

Step-by-step description

Read the environment state, not the message.
Repeat each task and report pass^k.
Average per task from n trials.
Match tool calls to a reference path.
Check arguments and single steps.
Mix code, model and human graders.

The outcome check reads the environment state, not the final message.
Repeated trials give pass^k, the chance that all k trials succeed.
The estimator averages per task, so it differs from a pooled success rate.
The trajectory check compares tool calls to a reference path.
Code, model and human graders each cover different checks.

In plain words

Think of a driving test. The examiner does not only ask whether you arrived. They watch whether you stopped at the lights, and a good school wants you to pass on most days, not once. The analogy breaks in one place. A driving test has one fixed route, while an agent task can have several valid paths and a simulated user who varies.

Some vocabulary, following Anthropic's guide to agent evals. A task is one test with defined inputs and success criteria, and each attempt at it is called a trial. A grader is logic that scores one aspect of the agent's performance. A trajectory is the ordered record of the agent's messages, tool calls and tool results in one trial. The harness is the code around the model that runs the loop and the tools. You evaluate the harness and the model together, not the bare model.

Step 1: Grade the outcome, not the final message

The outcome check reads the environment state, not the final message. Anthropic gives the example of a flight-booking agent that says the flight is booked, while the real test is whether a reservation exists in the environment's Structured Query Language (SQL) database (Anthropic, 2026).

τ-bench works this way. It compares the database at the end of a conversation with one annotated goal state, so every conversation that ends in that state passes, however it got there. It also checks that the replies contain required facts, such as the refund amount the user asked for.

Input: the final environment and the transcript. Output: pass or fail per check. The check is necessary but not sufficient. The τ-bench authors note that an agent might issue a return without explicit user confirmation, which breaks policy, so you add a check that the rules were followed.

Step 2: Repeat each task and report pass^k

Agents are non-deterministic, so one trial per task proves little. Repeated trials give pass^k, the chance that all k trials succeed. Its counterpart pass@k is the chance that at least one of k trials succeeds (Yao et al., 2024). Use pass@k when one good solution is enough, and pass^k when users need the agent to work every time.

The τ-bench authors report that gpt-4o with function calling solved about 61% of retail tasks in one trial (Yao et al., 2024, Table 2), but under 25% consistently across eight trials (Yao et al., 2024, abstract). Why does the number fall so fast? If trials were independent and each succeeded with probability 0.75 (illustrative), three successes in a row happen 0.4219 of the time. Eight successes in a row happen 0.1001 of the time at that rate.¹ Real tasks differ in difficulty, so treat this as intuition and not as a forecast.

Even a 75% per-trial agent succeeds eight times in a row only about one time in ten.CC BY 4.0 — source: How do you evaluate an AI agent, beyond just checking the final answer?

Text description of the diagram

Three successes in a row happen 0.4219 of the time at 0.75 per trial.
Eight successes in a row happen 0.1001 of the time at that rate.

Step 3: Estimate pass^k from n trials

With n trials of one task and c successes, the unbiased estimates are pass^k = C(c,k) / C(n,k) and pass@k = 1 − C(n−c,k) / C(n,k), averaged over tasks (Yao et al., 2024). Here C(a,b) counts the ways to choose b items from a.

Take one task with ten trials and eight successes (illustrative). Then pass^3 is 56 / 120, about 0.4667, while pass@3 is 1.¹

The same eight successes in ten trials give a pass@3 of 1 but a pass^3 of about 0.47, so the two metrics answer opposite questions.
Metric (illustrative, 8 of 10 trials succeed)	k = 1	k = 2	k = 3
pass^k	0.8	0.6222	0.4667
pass@k	0.8	0.9778	1

The estimator averages per task, so it differs from a pooled success rate. Take two illustrative tasks: an easy one with ten of ten successes and a hard one with four of ten. Mean pass^3 is 0.5167, while cubing the pooled rate of 0.7 would wrongly give 0.343.¹

Step 4: Compare the trajectory with a reference that allows more than one path

The trajectory check compares tool calls to a reference path. Exact match is simple, but LangChain's documentation warns that sometimes there can be multiple correct paths. Its trajectory-evaluation page lists match modes: exact order, any order, subset (no extra tools) and superset (at least the reference tools, extras allowed). Exact order is the strictest rule and fits steps that must happen in sequence. Any order accepts the same calls shuffled. No extra tools lets the agent skip a step but forbids additions, which suits efficiency checks. At least the reference tools allows additions but demands every reference call. The table compares tool names only; the evaluator can also compare tool-call arguments.

Below, the reference is get_reservation, check_fare_rules, cancel_reservation, book_flight. The agents are illustrative.¹

Only exact order rejects an agent that makes the right calls in a different order, and name-only matching passes an agent that used the right tools with a wrong argument.
Agent	Exact order (tool names only)	Any order (tool names only)	No extra tools (tool names only)	At least the reference tools (tool names only)
A: same calls, same order	pass	pass	pass	pass
B: same calls, swapped order	fail	pass	pass	pass
C: adds `search_flights`	fail	fail	fail	pass
D: skips `check_fare_rules`	fail	fail	pass	fail
E: right tools, wrong reservation	pass	pass	pass	pass

Step 5: Check tool arguments and single steps to find where it broke

Agent E shows the gap: name-only matching sees the tools selected, not the input passed to them. Match tool arguments too where the evaluator supports it, or add a state check that catches the wrong reservation. LangChain's documentation suggests giving a language model used as a judge the full trajectory plus a reference trajectory, which it calls the most challenging reference to compile.

Arguments matter. Of 36 gpt-4o failures on τ-retail, wrong argument made up 33.3%, wrong decision 25.0%, wrong information 22.2% and partially resolved 19.4% (Yao et al., 2024).

A single-step eval tests one decision, such as whether the agent picks the right tool. It is fast and pinpoints failures but does not capture the full agent (LangChain). Partial credit also helps when an agent takes some correct steps but misses the final answer (LangChain).

Step 6: Mix graders by what each check needs

Code, model and human graders each cover different checks. Anthropic describes code-based graders as fast, cheap, objective and reproducible, but brittle to valid variations. Model-based graders are flexible but non-deterministic and need calibration with human graders. Human graders are the gold standard but slow and expensive.

Anthropic's illustrative support task combines a state check (ticket resolved), a required tool-calls check, a transcript limit of ten turns and a language-model rubric for tone. It also reports a rigid grader failing a valid answer: a Claude Opus 4.5 model found a loophole in a flight-booking policy, failed the eval as written, yet served the user better. If a grader says no, read the transcript before blaming the agent.

Knowing it is half the job. Saying it is the other half.

Interview Clozo is an AI interview copilot for real interview rounds: it hears the question and puts a strong answer on your screen, and you say it in your own words. See the features or how it works.