Inference-Time Scaling of Verification: Self-Evolving Deep Research Agents via Test-Time Rubric-Guided Verification

AI summary

Overview

Research area: Deep Research Agents (DRAs) built on LLMs/VLMs — specifically automated verification, rubric-based reward modeling, and inference-time self-improvement of multi-step web-research agents.

Technical level: Advanced. The paper assumes familiarity with agentic pipelines (planning, web search, tool use), LLM-as-judge evaluation, supervised fine-tuning of open models, and test-time scaling concepts.

One-sentence scope: The paper builds a taxonomy of Deep Research Agent failures, turns it into a three-module rubric-guided verifier called DeepVerifier, and shows that repeatedly verifying and feeding corrective feedback back to an agent improves accuracy at test time without any additional training, while also releasing a 4,646-example SFT dataset (DeepVerifier-4K) to teach open models to do the same.

What This Paper Is About

Deep Research Agents often produce wrong answers because of bad searching, faulty reasoning, hallucinations, or failed tool calls, and in long-horizon tasks (the paper's collected trajectories average 8.2M tokens) human supervision at test time is infeasible. Rather than improving the agent by post-training its policy, the authors explore an alternative: let the agent self-improve by having a verifier check its output and feed back targeted, rubric-based corrections, iterating until the answer is accepted. The goal is a plug-and-play verification module that scales agent accuracy with feedback rounds and also yields a training dataset that teaches smaller open-source models to reflect on their own outputs.

Key Contributions

1. A DRA failure taxonomy, constructed automatically-then-human-verified. Starting from failure trajectories on WebAggregatorQA, the authors categorize failures into five major classes and thirteen sub-categories, and derive structured rubrics for outcome-based rewards. The most frequent class is Finding Sources, followed by Reasoning, then Problem Understanding, Action Errors, and Max Step Reached.

2. DeepVerifier, a rubrics-based outcome reward verifier exploiting the asymmetry of verification. It is a three-stage pipeline (decomposition agent, verification agent, judge agent) that decomposes holistic verification into small, targeted, evidence-checkable follow-up questions instead of re-solving the whole task. It outperforms vanilla agent-as-judge and LLM judge baselines by 12%–48% in meta-evaluation F1 score.

3. Inference-time scaling of verification for self-evolution. DeepVerifier plugs into a DRA during inference, produces rubric-based feedback, and drives iterative retries with no additional training. This yields 8%–11% accuracy gains on challenging subsets of GAIA and XBench-DeepResearch when powered by capable closed-source LLMs (the introduction states 8–11% on challenging GAIA subsets and 3–6% on XBench-DeepSearch).

4. DeepVerifier-4K and DeepVerifier-8B. A curated SFT dataset of 4,646 high-quality prompt-response pairs for DRA verification, produced by filtering and parsing 400 initial agent verification trajectories. Fine-tuning Qwen3-8B on it (mixed with the CK-Pro-8B training set) produces DeepVerifier-8B, which surpasses other open-source models after reflection on key benchmarks.

Main Findings

• Verification decomposition beats holistic judging. On GAIA-Web, DeepVerifier scores 75.00 precision / 71.43 recall / 75.56 accuracy / 73.17 F1. A decomposition-only variant ("−Verification", effectively an LLM judge) reaches 100.00 precision but only 14.29 recall, 60.00 accuracy, and 25.00 F1. A vanilla agent-as-judge ("−Decomposition") reaches 86.96 / 47.62 / 72.22 / 61.54. The authors attribute this to judges catching obvious failures (e.g., execution errors) while missing subtler reasoning or factual errors.

• Accuracy climbs with feedback rounds and peaks early. On GAIA-Web with Claude-3.7-Sonnet, accuracy goes 51.11 → 58.89 → 63.33 → 62.22 → 61.11 → 62.22 across rounds 0, 2, 4, 6, 8, 10, for a final gain of 11.11 and best gain of 12.22. On GAIA-Full, Claude-3.7-Sonnet goes 52.22 → 56.49 → 60.12 → 58.93 → 58.32 → 58.93, final gain 6.71, best gain 7.90.

• Gains generalize across backbones. GPT-4.1 on GAIA-Web moves from 28.89 at round 0 to a best of 32.22, and on GAIA-Full from 29.51 to a best of 32.53 (final gain 2.41, best gain 3.01). The reasoning/file-operations subset also improves for both Claude-3.7-Sonnet and GPT-4.1.

• The scaling effect transfers to other DRA benchmarks. On XBench-DeepSearch accuracy goes from 41.0 at 0 rounds to a best of 47.0 (round 2), ending at 44.0 at 10 rounds (final gain 3.0, best gain 6.0). On BrowseComp it goes from 5.0 to a best of 10.0, ending at 9.0 (final gain 4.0, best gain 5.0).

• Peaks are explained by asymmetric transition rates. Table 5 shows the correct-transition rate (incorrect → correct) is larger but decays quickly — 18.99, 9.33, 6.94, 8.45, 0.00, 1.45, 0.00, 0.00, 1.45, 0.00 percent across rounds 1–10 — while the regression rate (correct → incorrect) is smaller but persists — 12.79, 4.44, 4.30, 1.06, 3.03, 0.00, 0.00, 1.03, 0.00, 0.00 percent. Their interaction produces a peak around the fourth round.

• Reflection ability can be trained into open models. DeepVerifier-8B reaches 32.2% accuracy after reflection, a 5.5% improvement over its non-reflective result. CK-Pro-8B, trained only on the CK-Pro dataset, gains 2.6 points, while Qwen3-8B, trained on neither, shows minimal improvement. On the GAIA-Web split, DV-8B goes from 26.67 at round 0 to 33.33 at rounds 8 and 10 (final and best gain 6.67).

• Cost is bounded by design. The decomposition module narrows verification to ≤3 targeted follow-up questions, the loop terminates as soon as the verifier accepts the answer, and accuracy peaks around round 3–4, enabling practical early stopping. The authors note that iterative verification still adds inference steps and often extra tool calls.

• Taxonomy statistics. The trajectory corpus comprises 2,997 agent actions across 90 distinct tasks, with trajectories ranging from 2 to 156 steps (average 33.3), token counts from 18.7K to 60.0M (average 8.2M, total 738M, counted with the GPT-4o tokenizer), and a correct/incorrect ratio of 0.96. Annotation produced 555 error points, with 63.0% average overlap between the two annotators.

Methodology in Plain English

The authors first needed a vocabulary for "what goes wrong." They ran a representative open-source deep research agent (Cognitive Kernel-Pro, with Claude-3.7-Sonnet as the backbone) on the WebAggregatorQA benchmark, collected its trajectories, and had two research annotators compare the agent's execution against human reference solution traces to mark concrete error points. Clustering an initial subset of 50 error points and iterating produced a taxonomy of five major failure classes and thirteen subclasses, which in turn became a set of rubrics scored on four levels (Excellent / Good / Needs Improvement / Poor) per category.

DeepVerifier itself has three parts. The decomposition agent first summarizes a trajectory step-by-step by source and retrieved fact (necessary because trajectories average 8.2M tokens, far beyond any context window); it then scans the summary against the failure taxonomy to flag suspicious behaviors paired with probable error labels; and finally writes a small number of yes/no follow-up questions designed to confirm or refute the risky claims. The verification agent (CK-Pro, capable of web search, screenshotting, and code execution) answers those questions by retrieving evidence. The judge agent reads the task, the unverified answer, the trajectory summary, the potential-error list, and the follow-up answers, then gives a short explanation and a score from 1 to 4 (1 = entirely incorrect, 4 = entirely correct).

For self-evolution, the judge's prompt is modified to emit actionable retry instructions — including suggesting the correct answer when it is derivable from available information — and the agent repeats until accepted or a retry limit is hit. To build the dataset, the authors collected 400 answers and trajectories, ran DeepVerifier to produce verification trajectories, kept the true positives and true negatives (correct accepts of true answers and correct rejects of false ones), balanced them, and converted them into 4,646 prompt-response pairs. Evaluation uses precision, recall, accuracy, and F1 for verification (a true positive = verifier rejects a wrong answer; a true negative = verifier accepts a correct answer), with scores ≤2 treated as incorrect and ≥3 as correct during scaling experiments.

Why This Matters

The paper reframes agent improvement away from "train a better policy" toward "verify better at inference time" — and shows the payoff holds across model families and benchmarks. It also connects verification quality to concrete cost control, since the decomposition step replaces full re-solving with at most three targeted questions.

Real-world applications:

• Deep research and analyst assistants that must produce auditable, source-grounded answers rather than confident but wrong ones.
• Enterprise document and web research workflows where an automated checker can flag unsupported claims before output reaches a human.
• Multi-step tool-use and browsing agents (customer support, procurement, compliance research) where silent failure is more dangerous than an obvious error.
• Reward and feedback signal generation for reinforcement learning pipelines training agentic models.

Industry relevance: The taxonomy, the plug-and-play verifier, and the released DeepVerifier-4K dataset give teams a way to add a self-critique loop to an existing agent without retraining, and to bootstrap reflection ability into smaller open models. The authors explicitly frame the taxonomy and dataset as a foundation for reward signals in reflection-enhanced RL.

Future Directions

• Closing the precision–recall gap in the verifier. The paper attributes the performance peak around round 4 to the verifier's imperfect precision and recall: it fixes many wrong answers per round but also occasionally rejects correct ones. Improving verifier recall without sacrificing precision would likely push the peak later and higher.

• Reducing test-time cost and latency. The limitations section names this directly: while the decomposition module trims redundant problem-solving, iterative verification still adds inference steps and often extra tool calls. Efficient stopping criteria or cheaper verification models are open problems.

• Extending beyond the current benchmark set. The authors evaluate on GAIA (full and GAIA-web), XBench-DeepSearch, and BrowseComp. Whether the failure taxonomy — built from WebAggregatorQA trajectories — covers failure modes in other domains, modalities, or agent architectures is not established.

• Using verification as a reward signal for RL. The paper notes that reflection-enhanced reinforcement learning is gaining momentum and positions its taxonomy and DeepVerifier-4K as candidates for reliable self-verification and reward signals; turning the verifier into an RL reward and measuring the resulting policy improvements is left as future work.

• Addressing dependence on model capability. The authors note that feedback quality degrades when the underlying model is weak or lacks tool-use ability, and suggest DeepVerifier-4K SFT as partial mitigation.

Target Audience

Researchers and engineers working on LLM-based agents — particularly those building deep research, web browsing, or tool-use systems — will get the most from this paper, as will practitioners interested in LLM-as-judge evaluation, self-critique/Reflexion-style loops, rubric-based reward modeling, and test-time scaling. It is also relevant to teams training smaller open models for agentic reflection, since the paper releases both a dataset and a fine-tuned checkpoint. Readers looking for a first introduction to agent evaluation will find it demanding: it assumes comfort with benchmark metrics, ablation design, and multi-module agent architectures.

Note on reporting: the paper's truncated content includes the line "The training parameters are set as follows:" without the parameter values, so exact fine-tuning hyperparameters are not reported in the available text. Token counts, trajectory counts, and all accuracy/F1 values above are reproduced exactly as they appear in the paper. The abstract describes 8%–11% gains on GAIA and XBench-DeepResearch, while the introduction describes 8–11% on GAIA subsets and 3–6% on XBench-DeepSearch; both framings are as stated in the source, and the per-round tables show XBench-DeepSearch at 41.0 → 47.0 best (+6.0) and BrowseComp at 5.0 → 10.0 best (+5.0).

Authors’ abstract

Recent advances in Deep Research Agents (DRAs) are transforming automated knowledge discovery and problem-solving. While the majority of existing efforts focus on enhancing policy capabilities via post-training, we propose an alternative paradigm: self-evolving the agent's ability by iteratively verifying the policy model's outputs, guided by meticulously crafted rubrics. This approach gives rise to the inference-time scaling of verification, wherein an agent self-improves by evaluating its generated answers to produce iterative feedback and refinements. We derive the rubrics based on an automatically constructed DRA Failure Taxonomy, which systematically classifies agent failures into five major categories and thirteen sub-categories. We present DeepVerifier, a rubrics-based outcome reward verifier that leverages the asymmetry of verification and outperforms vanilla agent-as-judge and LLM judge baselines by 12%-48% in meta-evaluation F1 score. To enable practical self-evolution, DeepVerifier integrates as a plug-and-play module during test-time inference. The verifier produces detailed rubric-based feedback, which is fed back to the agent for iterative bootstrapping, refining responses without additional training. This test-time scaling delivers 8%-11% accuracy gains on challenging subsets of GAIA and XBench-DeepSearch when powered by capable closed-source LLMs. Finally, to support open-source advancement, we release DeepVerifier-4K, a curated supervised fine-tuning dataset of 4,646 high-quality agent steps focused on DRA verification. These examples emphasize reflection and self-critique, enabling open models to develop robust verification capabilities.