EvalResearchBench: Can AI Agents Build Their Own Feedback for Recursive Self-Improvement?

AI summary

Overview

Research area: AI agent evaluation, autonomous research, and recursive self-improvement — specifically whether agents can design the measurement instruments that drive their own improvement loops.

Technical level: Advanced. The paper assumes familiarity with benchmark design, LLM agent harnesses, and rank-correlation-style agreement metrics.

Scope: EvalResearchBench (ERB) is a benchmark in which LLM agents ("researchers") are given three hours and fixed API budgets to design, pilot-test, and freeze an executable evaluator across coding, co-work, and reasoning, which is then scored independently against 14 established target benchmarks and 13 candidate models.

What This Paper Is About

Recursive self-improvement depends on evaluation feedback to tell a system whether a change helped, but running large, complex benchmarks repeatedly is slow and expensive. Human experts get around this by hand-picking benchmark subsets or building cheaper proxy suites — the paper asks whether an AI agent can automate that design work itself, choosing tasks, writing graders, and revising them under a budget. EvalResearchBench formalizes this as "autonomous evaluation research" and measures how closely an agent-built evaluator can reproduce the scores and rankings of real target benchmarks it is not allowed to see.

Key Contributions

1. A new task formulation and benchmark for autonomous evaluation research. ERB defines the roles of researcher (an LLM agent in a harness), candidate (a model–execution configuration being measured), and target benchmark (the reference evaluation to be reproduced), with a strict freeze-then-assess protocol: tasks, grading code, and weights are locked before any assessment runs.

2. A budgeted, multi-domain research environment. Each episode spans coding, co-work, and reasoning simultaneously, with separate budgets for researcher inference ($100), shared development/helper testing ($100), independent assessment ($120 total cap, $30 per-candidate cap), and a three-hour deadline. The agent must submit at least 100 distinct scored questions with executable graders and fixed aggregation rules, and may not look up candidate identity or reference labels at scoring time.

3. Two complementary agreement metrics plus a reference-imposed ceiling. Score is 100 times Lin's concordance correlation coefficient between measured and reference scores (penalizing level and spread mismatch as well as order), while pairwise agreement (PA) measures the fraction of candidate pairs ordered correctly. The authors also compute an oracle PA ceiling from disagreements among the targets themselves, so results can be read against what any single evaluator could achieve.

4. A diagnostic analysis of what makes an agent-built evaluation good or cheap. Component ablation, nested-subset, and reweighting experiments separate task selection, question count, aggregation, and execution cost, and are paired with a human-designed public-task sampling baseline (Random120).

Main Findings

• No researcher wins everywhere. Across 9 researcher configurations and 13 candidates, Grok 4.7 leads coding, Astra leads co-work, and Opus 5 leads reasoning. Astra attains the highest overall Score (22.79 out of 100), while Grok 4.7 attains the highest overall PA (74.87%). Coding is the weakest domain for most evaluators.

• The best ordering is far below the ceiling. The best evaluators order about 75% of candidate pairs as the targets do, against a 91% ceiling: the oracle maximum is 91.20% overall (90.95% development, 94.59% sealed) because the target benchmarks themselves disagree on how to order the same candidates.

• Fitting visible references does not transfer. Fable 5.1 has the highest development Score (23.40) but not the highest sealed Score, where Grok 4.7 leads (26.14). Strong fit to the references used during development is not evidence of a broadly useful evaluator.

• A simple human-designed baseline is hard to beat. Random120 — 40 coding, 40 co-work, and 40 reasoning tasks drawn from 16 public sources with fixed quotas, graded with local graders and equal item weights — reaches an overall Score of 13.10 and PA of 68.29%. Seven of eight agent-designed evaluators exceed it in Score, but only four exceed it in PA. Its coding PA (75.25%) is higher than every agent-built evaluator's overall figure except Grok's.

• Low execution cost does not cost accuracy. Grok 4.7's recorded assessment spend is $8.63, versus $109.41 for Fable 5.1, $105.24 for Opus 5, and $118.02 for Fable 5 — yet Grok has the highest PA. Grok and Fable 5.1 submit similar question counts (107 and 110), so task count alone says little about recurring cost. The split comes from what a candidate must produce versus how the answer is judged: Fable 5.1 spends $33.97 on a rubric judge, Grok spends nothing there but relies on short, locally graded answers.

• More questions help, and stabilize. When nested, domain-stratified subsets of frozen suites are rescored on the same candidate panel, mean Score rises with subset size and both Score and PA fluctuate less. Small subsets can drop useful task families even when domain proportions are held fixed.

• Weights can throw away signal the suite already has. Grok assigns almost all co-work weight to six of its twenty co-work questions (93.87%), and three HLE tasks carry 3.60% of reasoning weight. Recomputing its scores with equal item weights on the same responses substantially improves coding and co-work Score, while overall PA barely changes — the frozen suite contained better measurement than its own weighting used.

• Adding harder tasks is not automatically better. K3 adds hard coding problems after near-ceiling pilots and Astra adds repository traces after reading upstream software tasks, but removing those groups has only modest effects relative to same-domain, same-size random deletions (K3's addition improves Score but slightly reduces PA; Astra's leaves PA unchanged). Astra's spreadsheet tasks, by contrast, contribute strongly on both co-work metrics.

• The agents genuinely iterate — and still ship flaws. Researchers repair prompt–rubric mismatches, fix invalid examples and numerical answers, and adjust answer allowances after truncated pilots. Final evaluators can nevertheless truncate answers, exhaust the assessment budget, or let a few heavily weighted questions dominate a domain score. Fable 5 ran out of assessment budget after 20 of 39 candidate-domain reports; its imputed sensitivity scenario (19 missing scores set to 100%) produces negative Scores and is reported separately from the eight fully observed submissions.

• Uninformative designs are correctly punished. Assigning every candidate 0.5 in every domain with no model calls yields Score = 0 and PA = 50.00% across the same 14 target panels.

• Reference disagreement is measurable. For ProgramBench (Vals) and SWE-bench Multilingual, which share six reference-bearing candidates, the raw-score error floor from their disagreement alone is 0.4356 (reference means 0.0075 and 0.876). Oracle RMSE floors across all targets are 0.303 for coding, 0.128 for co-work, and 0.244 for reasoning.

• A scale-compatibility check reorders the leaderboard. On an eight-target subset chosen without reading any researcher score, Fable 5.1 leads with Score 33.47 and Grok leads PA at 75.06% — so the top Score depends on the target panel, while Grok's ordering lead is more robust.

Methodology in Plain English

The authors set up a competition between an AI agent and a hard measurement problem. Each of nine researcher agents (GPT-6 Astra, GPT-5.6 Sol, GLM-5.3, DeepSeek V4 Pro, Kimi K3, Fable 5.1, Opus 5, Grok 4.7, Fable 5) is dropped into a coding-agent harness — Codex for the GPT and Grok researchers, Claude Code for the others — and given one episode plus a fixed stack of information: descriptions of the target benchmarks, some original tasks and grading resources, historical execution traces, and reference scores for four "development" models. Nine further "sealed" models and eight sealed target benchmarks stay hidden.

Within a three-hour limit and $200 of metered spending, the agent explores, writes and edits tasks, writes graders, runs pilots, sees candidate responses and how well its scores line up with the visible references, and revises. It then freezes its evaluator — at least 100 scored questions with executable graders and fixed weights — producing one score per domain.

Judging happens afterward and is never fed back. The frozen program is run on all 13 candidates under a separate $120 cap, and its scores are compared with both the visible references (development targets) and the hidden ones (sealed targets). Two numbers summarize each comparison: Score, based on Lin's concordance correlation coefficient, which rewards getting the levels, the spread, and the ordering right; and pairwise agreement, which asks, for every pair of candidates the target separates, whether the agent's evaluator put them in the same order, giving half credit for ties. Reference benchmarks are normalized to a common scale (percentages divided by 100; Elo converted with a fixed offset of 0.00 and divisor of 1640.35).

The analysis then isolates causes. The authors delete one task group from a frozen suite at a time and compare the resulting drop against random deletions of the same size and domain; they rescore nested subsets to see how question count affects both accuracy and stability; they reweight Grok's frozen responses to test whether the chosen weights were the problem; and they split recorded costs into researcher inference, development testing, and frozen assessment. Two oracle calculations — the best achievable PA and the best achievable RMSE given how much the target benchmarks disagree with each other — turn raw results into a fraction of what is realistically attainable.

Why This Matters

Research impact. Every self-improvement loop, whether it is post-training, program evolution, or agent redesign, assumes a trustworthy signal telling it whether a change helped. Most work in this space takes that signal as given: PostTrainBench fixes the harness, RSIBench-Data fixes the evaluation service, AlphaEvolve takes a user-supplied evaluation function. ERB is the first to isolate the step where the signal itself is built, and its main result is sobering — agents can build useful evaluators, but they land near 75% ordering agreement against a 91% ceiling, and they do not reliably beat a well-constructed sample of existing public tasks. It also gives the field a way to quantify how much of the remaining gap is the agent's fault and how much is conflicting references.

Real-world applications:

• Cheap model-selection pipelines: teams deciding which model to deploy could have an agent build a compact suite that reproduces a large leaderboard's ranking, at roughly $9 of assessment spend per full candidate panel rather than $100+.
• Domain-specific proxies for expensive agentic benchmarks: evaluations requiring long tool-using trajectories or specialized environments are the costly case ERB targets, mirroring the tinyBenchmarks motivation.
• Continuous regression testing for agent developers: the construction-cost / execution-cost split gives a template for iterating on evaluation cheaply while keeping the recurring per-candidate cost low.
• Automated audit of in-house evaluation suites: the deletion, subset-size, and reweighting analyses are directly reusable as diagnostics for whether a benchmark is actually discriminating between models or just at ceiling.

Industry relevance. The strongest practical finding is the cost result: the evaluator with the lowest recorded execution spend ($8.63) achieved the highest pairwise agreement, while two competitors spent over $100 with worse ordering. Product and platform teams care about recurring evaluation cost because it is paid on every new model or checkpoint, and the "short locally graded answers versus long deliverables scored by a rubric judge" distinction is an actionable design lever. The finding that equal weighting improved a frozen suite's Score substantially suggests many deployed eval suites underuse signal they already collected.

Future Directions

• Closing the gap to the reference ceiling. The 75% versus 91% shortfall leaves open whether better researchers, longer budgets, or more development references would close it — and whether the ordering conflicts inside the target panel itself should be resolved by better target curation rather than better agents.

• Transfer beyond the development references. Fable 5.1 leading on development but not on sealed targets makes transfer the central open problem; it is not reported whether any intervention (for example, deliberately holding out part of the visible reference set) improves sealed performance.

• Fixing reweighting and budget exhaustion. Since equal weights substantially improved Grok's frozen suite and Fable 5 exhausted the assessment cap, the obvious next step is whether agents can be taught to check task contribution and budget headroom before freezing — the paper diagnoses both failures but does not test a remedy.

• Broadening domains and reference sources. The study covers three domains, 13 candidates, and 14 targets with fixed public reference snapshots; it is not reported how the conclusions extend to other capability areas, to targets without published score snapshots, or to references that disagree even more than these do.

Target Audience

Researchers and engineers working on LLM agent evaluation, benchmark construction, and self-improving systems will get the most from this paper, particularly those who build evaluation harnesses or run training loops that depend on benchmark feedback. It is also relevant to applied teams making model-selection decisions who want a principled way to assess whether a cheaper proxy suite really reproduces a full leaderboard. The advanced statistical framing (concordance correlation, oracle ceilings, RMSE floors) assumes a reader comfortable with evaluation metrics, but the core narrative — agents can build decent evaluators, not great ones, and cheap can beat expensive — is accessible to anyone who has maintained a benchmark.

Authors’ abstract

Recursive self-improvement (RSI) relies on evaluation feedback to assess progress and guide further research, yet repeatedly running complex benchmarks is costly and slows iteration. Human experts reduce this cost by selecting benchmark subsets or designing compact suites. We ask whether AI agents can automate this design process and introduce EvalResearchBench (ERB), a benchmark for autonomous evaluation research. Given target materials, development references, candidate APIs, and fixed time and API budgets, an agent called the researcher selects or synthesizes tasks, implements graders, and revises them in pilot tests before freezing an executable evaluator for coding, co-work, and reasoning. We study 9 researchers and 13 candidate models and compare each frozen evaluator with 14 target benchmarks on score concordance and pairwise agreement. The best evaluators order about 75\% of candidate pairs as the targets do, below the 91\% ceiling set by disagreements among the targets. No researcher leads on every metric, and the best evaluator on development targets is not the best on sealed targets hidden from the researcher. A human-designed sample of public tasks remains a strong baseline, and the evaluator with the lowest recorded execution cost attains the highest pairwise agreement. Agents repair tasks and graders through pilot feedback, yet their evaluators can still truncate answers, exhaust the evaluation budget, or let a few questions dominate a domain score.