Evidence-First Agentic Verification

AI summary

Overview

Research area: AI agent evaluation — specifically rubric-based verification of long-horizon, tool-using language-model agents, with implications for reinforcement learning (RL) rewards.

Technical level: Advanced. The paper assumes familiarity with LLM-as-a-judge evaluation, agentic tool-use traces, rubric grading, and reward hacking.

Scope: The paper introduces Sauron, an "evidence-first" agentic verifier that independently validates claims against an agent's final environment state and execution trace before issuing rubric verdicts, and evaluates it on three benchmarks (PA₁₀₀, PA_Fault, TRAIL) against four baseline verifiers across eight backbone models.

What This Paper Is About

Agents that perform long, multi-step tasks are typically graded by LLM judges against a rubric of criteria, but the evidence needed to grade them is scattered across the final environment state and the execution trace — and a judge may miss it or simply believe the agent's own claim of success. When such verdicts are used as RL rewards, these errors reward convincing-but-false behavior, a form of reward hacking. The paper's goal is a verifier that refuses to use any claim until it has been independently checked against its original source, while keeping the cost of that investigation lower than existing agentic judges.

Key Contributions

1. Sauron, an evidence-first agentic verifier. It separates evidence gathering from verification: an investigator queries state and trace and proposes claims ("evidence atoms"), a separate evidence verifier checks each claim against its cited source, and a verdict finalizer accepts a verdict only when verified claims support it. Every verdict carries an auditable receipt of its supporting evidence.

2. Bounded evidence retrieval and joint investigation. The full trace and state stay outside the verifier's context; size-capped broad and targeted queries fetch only what each criterion needs, and criteria likely to share evidence are clustered so one investigation session serves several criteria, reducing context use and redundant work.

3. An extensive three-benchmark evaluation. PA₁₀₀ (100 episodes, 2,586 human-labeled criteria, eight backbones), PA_Fault (637 fault-control pairs across 13 fault families, several simulating reward hacking), and TRAIL (real agent errors), with mechanism ablations and a manual analysis of residual disagreements with human judgments.

4. A paired fault-injection suite (PA_Fault). It measures detection while penalizing false alarms, and includes full-trace variants of trace-blind baselines to separate the effect of trace access from how verifiers use it.

Main Findings

• Best F1 on every backbone. Across eight backbones on PA₁₀₀, Sauron achieves the highest F1 of five verifiers while costing less than Gandalf, the strongest baseline; averaged across models it leads Gandalf by 0.03 F1 at roughly 20% lower cost.

• Headline result with GPT-5.6 Sol. Sauron reaches 0.96 F1 versus Gandalf's 0.92, and 0.94 versus 0.89 MCC, at 23% lower cost ($297.69 vs $388.25 per full run), improving both precision (0.94 vs 0.90) and recall (0.97 vs 0.94).

• The lead widens on process criteria. On the 411 PA_Proc criteria, Sauron gains 0.07 F1 over Gandalf (0.97 vs 0.90), a larger lead than on the 2,175 PA_Out outcome criteria (0.96 vs 0.93); the gap is larger on process than outcome criteria for all eight backbones.

• Cheaper and faster in practice. Sauron reads fewer than half as many input tokens as Gandalf with GPT-5.6 Sol; on a 10-episode slice it finishes in 17 minutes versus Gandalf's 71 when grading one episode at a time, and remains 2.1× faster with eight episodes graded concurrently.

• Snapshot and text-only verifiers lag. Archipelago trails Sauron by 0.04 F1 at 6.9× the cost ($2,058); Autorubric, which judges from the agent's response alone, never exceeds 0.70 F1 on any backbone.

• Strong fault detection with few false alarms. On PA_Fault, Sauron reaches Δ_fault = 0.95, detecting 97% of injected failures while flagging only 2% of clean controls, at $0.21 and 30 seconds per variant.

• Trace access alone is not enough. With the full trace, Gandalf^trace reaches Δ_fault = 0.67 and Autorubric^trace 0.71 — at least 0.24 below Sauron — and Gandalf^trace costs 4.7× as much per variant and takes twice as long. The trace-blind defaults score 0.09 and −0.17.

• Large gaps on state–trace conflicts. On the state mismatch family, Gandalf^trace and Autorubric^trace score −0.27 and −0.94 (negative meaning they flag clean controls more often than faulty variants) versus Sauron's 0.90; on silent write failures they score −0.10 and −0.20 versus Sauron's 1.00.

• Leads on real error localization (TRAIL). With GPT-5.6 Sol on GAIA, Sauron beats the best baseline by 0.07 wF1, 0.18 location accuracy, and 0.19 joint accuracy (0.53/0.73/0.44 vs Gandalf 0.46/0.53/0.25 and Judge_TRAIL 0.46/0.55/0.22), and leads on SWE-bench Lite as well.

• Cost caveat on TRAIL. Sauron costs two to three times the baselines there ($841 vs $441 and $405), because ruling out an error category can require examining the entire trace; however, Sauron with GPT-5.6 Terra outperforms both Sol-backed baselines on every metric at roughly half their cost ($202).

• What ablations show (PA_Abl, 860 criteria, GPT-5.6 Sol). A trace-blind Sauron loses 0.15 F1 (largest drop); putting the full trace in context gains only 0.01 F1 at 3.03× cost and 2.65× turns; disabling claim verification loses 0.03 F1 while cutting cost 40%; dropping the faithfulness checker alone produces the same 0.03 drop; removing clustering loses 0.01 F1 while raising cost 63% and turns 2.55×; removing the gist leaves F1 unchanged but raises cost 15%.

• Where it disagrees with humans. Of all 65 disagreements on the 2,586 PA₁₀₀ criteria, 48 (nearly three-quarters) came from Sauron reading a criterion more strictly than annotators — 34 marking a criterion unmet over literal wording or a single subclause, 14 counting an action as violating a negative criterion. In all but three cases the disagreement concerned how to read the criterion, not what the evidence showed.

Methodology in Plain English

For each task, Sauron receives the user request, the agent's final response, the final environment state and artifacts, the execution trace, and the rubric criteria. A small, cheap model (GPT-5.6 Luna) does two planning steps: it groups criteria that probably need the same evidence into clusters (default maximum 12 criteria), with a validation-and-repair pass ensuring each criterion appears exactly once, and it writes a short "task gist" briefing that suggests where to look. The gist is advisory only and can never serve as evidence.

Each cluster then runs as a parallel investigation session. The investigator cannot see the full trace or state; it must ask for pieces through bounded queries — broad queries to locate things, targeted ones to read specific values, artifact regions, or short trace windows — with per-query content caps and per-session budgets. It can also run code (e.g., openpyxl on a spreadsheet copy, pdfplumber on a deliverable PDF) for mechanical checks.

As it finds facts, the investigator proposes claims, each packaged as an "evidence atom" with a claim, an evidence type, and a source locator. The evidence verifier fetches the cited content and applies the appropriate check: exact claims are compared mechanically with the stored value, while interpretive claims go to an isolated faithfulness checker that sees only the claim and the passage (to reduce bias from the investigator's prior reasoning). Claims whose source cannot be retrieved are rejected outright. Rejected atoms are returned with a reason so the investigator can repair them, and accepted and rejected atoms alike are logged in the cluster's shared evidence store.

The investigator then proposes a verdict, which a separate finalizer accepts only if the verified atoms actually support it. Each criterion gets its own binary met/unmet verdict and its own receipt, even though the investigation was shared. Sessions end when all criteria are decided or the budget runs out — unresolved criteria default to unmet. Evaluation compares Sauron to Gandalf, Agent-as-a-Judge, Archipelago, and Autorubric on identical fixed trajectories with the same backbone, scoring with F1 (unmet as the positive class) and MCC, plus a paired fault-injection design and TRAIL's wF1, location accuracy, and joint accuracy.

Why This Matters

Impact on research. The paper reframes verification as an investigation with audit rather than a single judgment over a supplied context, and shows empirically that giving a judge trace access does not fix its errors — a useful corrective to work that treats access as the bottleneck. It also connects claim-level factuality checking and process supervision to rubric grading in a concrete architecture, and provides a fault-injection protocol that separates detection from false alarms, which most verifier papers do not measure.

Real-world applications:

• RL training for agents. Rubric verdicts are used as rewards; a verifier that refuses unsupported success reports reduces the incentive to reward-hack, which the paper argues (but does not yet test) should improve trained policies.
• QA of agent-produced deliverables. Financial reports, spreadsheets, and slide decks can be checked for internal consistency — e.g., a saved total that does not match retrieved account data — with a receipt showing exactly which cell or trace event proved it.
• Auditing and compliance. The per-verdict receipt gives a human reviewer a defensible evidence trail for why a criterion passed or failed.
• Agent debugging. On TRAIL-style workloads, the verifier's cited spans localize real errors in long trajectories, which can speed up post-mortems.

Industry relevance. The quality–cost frontier is the practical selling point: Sauron beats the strongest baseline on quality while costing less on PA₁₀₀, and the ablation showing that dumping the full trace into context gains 0.01 F1 at 3× cost is a direct argument against naive "just show the judge everything" pipelines. The authors are at Scale AI, and the code is linked from a public repository.

Future Directions

• Use Sauron as an actual RL reward provider. The paper explicitly leaves this for future work, proposing to measure effects on both task performance and reward hacking.
• Robustness to policies optimized against the verifier. The authors note that their controlled fault injections do not establish robustness to an agent trained to exploit Sauron, and that agent-authored content in state or trace (text written to sway the verifier) can still mislead model-based checks.
• Broadening coverage beyond the current scope. Evaluations are English-only, PA₁₀₀ and PA_Fault come from a single agent (Opus 4.8) in the personal-assistant domain, and self-preference (verifiers favoring agents from their own model family) is not systematically tested.
• Reducing cost where broad coverage is required. Bounded queries save less on tasks like TRAIL error detection that need the whole trajectory; cheaper or better-targeted search, and cheaper faithfulness checking (which accounts for the 0.03 F1 / 40% cost trade-off), are open engineering questions.

Target Audience

Researchers and engineers building agent evaluation harnesses or RL training loops for tool-using agents; practitioners running LLM-as-a-judge pipelines who need to justify verdicts with evidence; and readers interested in reward hacking, process supervision, and long-context evidence retrieval. Readers new to agentic evaluation will find the architecture description readable, but the benchmark and cost analysis assumes familiarity with F1/MCC scoring and judge baselines.

Authors’ abstract

Long-horizon, semi-verifiable agentic tasks often rely on rubric-based evaluation, but the evidence needed to assess these tasks can be spread across the environment state and the execution trace. A judge may overlook this evidence or accept unsupported claims, including an agent's own reports of success. When rubric verdicts serve as reinforcement-learning rewards, such errors can reinforce undesirable behavior. We introduce Sauron, an evidence-first agentic verifier that investigates both state and trace and relies on a claim only after independently validating it against its source. Bounded queries and joint investigation of related criteria reduce context use and redundant work, and each verdict carries an auditable receipt of its supporting evidence. Across 2,586 human-labeled criteria from 100 personal-assistant episodes, Sauron achieves the highest F1 of five verifiers on all eight backbones tested while costing less than Gandalf, the strongest baseline. With GPT-5.6 Sol, it reaches 0.96 F1 versus Gandalf's 0.92 at 23% lower cost, with a wider margin on process criteria. On a paired fault-injection suite covering 13 families, several of which simulate reward hacking, Sauron detects 97% of injected failures while flagging only 2% of clean controls, outperforming baselines even when they are given the full trace. On TRAIL, a benchmark for detecting and localizing real agent errors, Sauron leads both Gandalf and TRAIL's own judge on all three quality metrics.