Evaluating Agent Outcomes
Unit tests can verify a parser or permission check, but an agent also needs outcome evaluation: did it complete the task, use valid evidence and respect constraints?
#Build a regression set
Include ordinary requests, missing evidence, conflicting sources, malformed tool input, unavailable services and permission denial. Store input, expected behavior, allowed evidence and an evaluation rubric. Keep a held-out set so prompt edits do not simply memorize the examples.
Separate four layers: final output, action sequence, retrieval and business result. A polite answer can hide a failed action. A correct tool call can still produce an unusable final document.
#Use deterministic checks first
Validate schemas, required fields, source IDs and permissions with ordinary code. For a policy assistant, check that an absent rule produces an unresolved result rather than a fabricated number.
An illustrative rubric:
| Criterion | Pass condition |
|---|---|
| Support | Every factual policy claim maps to an authorized passage |
| Missing information | Unknowns remain explicit |
| Action integrity | No write occurs without the required authorization |
| Result | The complete requested artifact can be opened |
#Model judges are fallible
LLM-as-a-judge can help review relevance or support at scale, and RAGAS-style metrics can organize retrieval evaluation. Neither replaces calibration against human labels. Judges can prefer fluent but wrong answers, inherit prompt injection from examples or disagree across runs.
Record judge model, rubric and prompt version. Inspect disagreements and report uncertainty. Judge calls can incur additional cost and need the same data handling and authorization controls as generation.
#Make changes comparable
Run baseline and candidate against the same dataset and environment. Report case counts and failures alongside averages, latency and usage. Do not publish invented scores or copy a sample metric as if it were a measured result. Keep representative failing cases so the next change addresses a concrete defect.
