Agents Lookup

Learn / Practice

Measuring whether an agent is working

How to build a small test set, pick graders, track the right numbers in production, and tell a lucky agent from a reliable one.

Updated 2026-11-19. "(source N)" refers to the numbered list at the end.

Why gut feel is not enough

Most agent projects are judged by someone trying it a few times and saying it seems fine. Then a new model comes out, or someone edits the prompt, and nobody can say whether things got better or worse.

Quality is the biggest reason agents fail to ship. In LangChain's practitioner survey, about a third of respondents named it as their main barrier, more than speed or cost (source 2). The fix is a habit, not a product: write down what good looks like, test against it, and keep testing.

Step 1: define success in numbers

Anthropic's guidance says good success criteria are specific, measurable, achievable and relevant (source 5). Accurate replies is too vague. Ninety per cent of refund requests get the correct decision, and none promise a refund the policy does not allow, is testable.

Most agents need several measures at once (source 5). A support agent might be judged on:

Step 2: build a small test set

You do not need hundreds of tests to start. Anthropic's engineering team says 20 to 50 small tasks taken from things that actually broke is a good start (source 1). Pull them from support tickets, bug reports and the checks you already do by hand.

Make each task clear enough that two experts would agree on pass or fail, and write a reference answer for each (source 1). Include cases where the agent should act and cases where it should refuse or escalate, so you do not train it to always say yes.

OpenAI gives the same advice for its tools: use test data that looks like real traffic, include human-checked answers, start small and grow (source 4).

Step 3: choose your graders

A grader is whatever scores a result. Anthropic describes three kinds (source 1).

Which grader when

The advice is to use code checks where you can, model graders where you must, and people sparingly (source 1). LangChain's survey shows teams already mix them: 59.8% use human review and 53.3% use model graders (source 2).

Grade the outcome, not the exact path. If the agent found a different but valid way to solve the task, it should pass (source 1).

Step 4: test for consistency

Agents give different answers on different runs. One success proves little. Run each task several times.

Two measures help (source 1). Pass@k is the chance of at least one success in k tries, which suits work where a person picks the best of several drafts. Pass^k is the chance that all k tries succeed, which is what matters for anything customer-facing. An agent that succeeds 75% of the time passes three runs in a row only about 42% of the time.

The pass^k idea comes from the tau-bench research benchmark. Its authors found even strong agents of the time were inconsistent, with pass^8 below 25% on a retail task set (source 3).

Step 5: keep two kinds of suite

Anthropic separates capability tests from regression tests (source 1). Capability tests are hard tasks the agent cannot do yet; they should start with a low pass rate. Regression tests are things it already does well; they should stay near 100%. When a capability test passes reliably, move it into the regression suite.

Run the regression suite before every prompt change, model swap or tool change.

Step 6: read the transcripts

Scores lie when graders break. Anthropic warns that a 0% pass rate across many runs usually means a broken task, not a hopeless agent, and that no score should be trusted until someone has read the details (source 1). Read a handful of transcripts every week.

In production: watch the live numbers

Testing before launch is half the job. LangChain found 52.4% of teams run offline tests on fixed data and 37.3% score live traffic too (source 2). Among teams with agents in production, 94% have some monitoring and 71.5% have full tracing of each step (source 2).

Useful live numbers for a small business include: share of tasks completed without a person, share escalated, share a person had to correct, average cost per task, and complaints per hundred tasks.

Keep your tests portable

Store test cases in your own files, not only inside a vendor's dashboard. OpenAI's hosted Evals platform, for example, goes read-only on 31 October 2026 and shuts down on 30 November 2026 (source 4). Tools change; your test set is the asset that lasts.

Sources

Written by Agents Lookup in our own words. Facts come from these pages: