Recoverability and Verifier Strength: Two Measured Preconditions for Unattended Agent Self-Improvement

AI summary

Overview

Research area: Agentic AI platforms — specifically the evaluation, admission control, and rollback of automated changes to deployed LLM agents (a topic at the intersection of ML platforms, LLM-as-judge evaluation, and deployment engineering).

Technical level: Intermediate. Readers should be comfortable with agent benchmarks, LLM judges, and basic correlation/statistics, but the paper's core argument is stated plainly enough to follow without deep ML background.

Scope: The paper defines two measurable preconditions — verifier strength (σ, an MCC-based agreement score with a trusted reference verifier) and recoverability (re-approval g, restoration fidelity ρ, blast radius β) — proposes a rule that permits unattended agent changes only when both are adequate, describes a platform that enforces the rule, and tests the rule's assumptions in a case study of the platform's own LLM-judge gate plus controlled experiments on τ²-bench.

What This Paper Is About

Agent platforms are beginning to repair their own agents without a human author: they read production execution traces, derive a change (a prompt edit, a tool-definition fix, a skill update, or a fine-tune), have an automated verifier approve it, and ship it. The open question is how much of that loop can safely run unattended. The authors argue the answer depends on two quantities a platform can measure — how much the verifier's verdicts can be trusted, and how cheaply a bad change can be undone — and they build and test a rule that refuses autonomy whenever either is missing.

Key Contributions

1. Two measurable preconditions plus an autonomy rule. Verifier strength σ(v) = MCC(v, v*) on a trajectory set C, bounded in [−1, 1], chosen over accuracy so that a verifier that always gives the same answer scores exactly 0 rather than looking respectable. Recoverability is captured by three numbers: re-approval g(ι) ∈ {0,1} (does the undo need a fresh verifier approval?), restoration fidelity ρ(ι) (share of probe tasks with the same outcome after undo as before the change), and blast radius β(ι) (share of probe tasks whose outcome changed while the change was deployed, which no undo repairs). A change may go live unattended only if g(ι) = 0 and σ(v) ≥ σ_min; if either quantity is unmeasured, the platform refuses.
2. A platform that enforces the rule structurally. Changes are typed by recoverability across four resources — model weights, system prompt, tool definitions, skills — evaluation acts as a non-skippable admission gate, verifiers are declared resources carrying their measured σ, and refusal is recorded as its own third outcome (distinct from admit and reject) together with the missing precondition.
3. A case study documenting "silent gate collapse" in the platform's own production gate, with two cheap acceptance tests that catch it: delete the signal (remove the property the gate claims to measure and check the score changes) and check the guarantees (verify that request-level settings such as forced tool calls and output formats actually took effect).
4. Controlled τ²-bench experiments that vary verifier strength and recoverability separately, using the benchmark's configurable five-check task scorer so one set of trajectories can be re-scored under many verifiers.

Main Findings

• A production gate can look healthy while measuring nothing. On 172 settings of 10 runs each (1,720 calls, 1,718 returned a score), the deployed diversity gate returned 0.250 in all 40 runs of its production setting with zero spread — exactly what it returns when every mention of imbalance is removed from the prompt. Because 98 of 172 settings do vary, this is not an artifact of greedy decoding.
• Collapse happened in 9 of 37 configurations, all on the deployed (smallest) judge. The count alone is not conclusive, since that judge ran 33 of 37 configurations (Fisher p = 0.55); the matched comparison is: on identical inputs the deployed judge stays flat while both larger judges track imbalance (p < 10⁻¹¹ each). Collapse also depended on the agent.
• A checkable bound fixes the gate. Rewording the criterion as "if the largest seed exceeds 50%, score at most 0.10" held in all 16 settings where the largest seed exceeded 50% — 30 of 30 runs versus 0 of 30 with the original wording on matched inputs (Fisher p < 10⁻¹⁶).
• Real verifiers sit far from perfect. On 80 airline test trajectories (20 tasks × 4 trials), the LLM judge reached σ = 0.70 against the program check (Fisher p < 10⁻⁹) but with a task-resampled interval of [0.36, 1.00]; its 4 false admits came from a single task and its 8 false rejects from two, repeated across trials. A constant admit-all verifier scores exactly 0 while passing all 42 failing trajectories. Reference-action match scored σ = 0.67 with nearly the same σ but opposite error mix (α = 0.21, φ = 0.12 versus 0.10 and 0.21 for the LLM judge).
• Weaker verifiers picked worse changes, but the effect was not significant. Regret rose from 0.000 at σ = 1 to 0.067 at σ ≈ 0, as predicted, but the fitted slope (−0.065, CI [−0.234, 0.120]) includes zero (one-sided p = 0.23); resolving it would take about 220 held-out tasks, eleven times the 20 used. The candidate pool is "field-limited": the best candidate is only 0.069 above the average, so the observed 0.067 is already 98% of the maximum possible loss. Retail showed no trend (slope +0.008, CI [−0.115, 0.109]) and is untested rather than failed.
• Noise floors matter. A null change (no change at all) gave ρ₀ = 0.95 (CI [0.89, 1.00]) on the 20 test tasks and β₀ = 0.08 (CI [0.02, 0.16]) on all 50 airline tasks with four runs per task.
• Only a stored copy restores behavior. Restoring a saved prompt gave ρ − ρ₀ = 0.000 (CI [−0.038, 0.038]), equivalent to zero within ±0.05 (TOST p = 0.007). Three regenerated undos landed at ρ of 0.75, 0.83 and 0.90 against a training-split floor of 0.98 — 0.23, 0.15 and 0.08 below it, and 0.16 below together (CI [0.07, 0.27], p = 0.0001). The gate refused all three, as the rule requires.
• Admitted changes reach tasks that rollback cannot undo. Three admitted changes with complete data deployed on all 50 probe tasks altered the outcome of 17% of tasks beyond the noise floor (β − β₀ = 0.17, CI [0.08, 0.27], one-sided p = 0.0002); each was significant on its own after Holm correction. Flips went both ways in nearly equal numbers (18 up, 20 down), so the average pass rate barely moved even though roughly one task in six changed outcome.
• σ_min does not transfer. The strength below which weak verifiers start to hurt moved by ±0.15 with the number of simulations the gate averages — 0.41 with 60 runs per candidate, 0.56 with 30, 0.49 with 15 — so a threshold calibrated on one benchmark would not carry over.
• The loop overstates its own gain even with a perfect verifier (0.20 gap; one-sided p = 0.08, not replicated in retail), split between a training/test task difference (0.09) and the winner's curse (0.11), neither individually significant.
• The rule's placement: prompt, tool-schema and skill edits run unattended; weight updates and fine-tunes without a kept checkpoint require human sign-off; a training-data filter with no workable verifier ships disabled.

Methodology in Plain English

The authors first state the rule as a design constraint, then build a platform that cannot bypass it, then test the assumptions the rule rests on in two settings.

For the gate case study, they took their own production "diversity gate" — an LLM judge that scores a training set from 0 to 1 and blocks low-diversity sets from being used for fine-tuning — and fed it training sets whose imbalance they set by hand while holding everything else fixed. They swept 3 judge models, 4 prompt variants, 4 imbalance levels, breadths from 3 to 12, and two agent configurations, running each setting 10 times. A configuration counted as collapsed if it gave the same score at all four imbalance levels. They then compared judges on identical inputs and tried a rewording of the prompt that states the criterion as a numeric bound.

For the controlled experiments they used τ²-bench, whose task score is the product of a configurable subset of five checks. That configurability is the key trick: changing the subset changes the verifier while leaving tasks, agent and environment fixed, and because scoring is separate from running, one set of trajectories can be re-scored under every verifier. The airline domain provided 50 tasks split 30 training / 20 testing with natural-language assertions on every task. Experiment A tested verifier strength — A1 measured σ for real verifiers on one set of 80 trajectories, and A2 created verifiers of known strength by flipping reference verdicts with probability ε from 0 to 0.5 in 0.05 steps, averaging over 400 random corruptions per point. Experiment B fixed the verifier at the reference and compared restoring a stored prompt copy (g = 0) against asking the optimizer to regenerate an undo (g = 1), plus deploying admitted changes to measure blast radius. Each agent ran twice per training task and four times per test task; intervals are 95% task-resampling bootstrap intervals (2,000 resamples in experiment A, 20,000 elsewhere), one-sided p-values where the hypothesis fixes a direction, with Holm correction across candidates.

Why This Matters

The paper reframes agent self-improvement from "can the loop produce a good change?" to "which parts of the loop may run without a human, and what must be measured first?" The distinctive move is treating verifier reliability and rollback cost as measured platform properties with recorded numbers, rather than as assumptions baked into a pipeline. The silent-collapse finding is the sharpest practical warning: a judge that returns a confident-looking constant and raises no error is indistinguishable from a working judge unless someone deliberately removes the signal and checks.

Real-world applications:

• Customer-service and IT agent platforms that continuously patch prompts, tool schemas and skills from production traces — the paper's own setting — where the rule lets routine edits auto-promote while gating weight updates behind sign-off.
• Automated fine-tuning pipelines that need a defensible check on LLM-judge data-quality or diversity gates before a training set is used.
• Evaluation and observability tooling, where "delete the signal" and "check the guarantees" become regression tests run whenever the judge model, prompt, or request format changes.
• Incident and change management for agents, where blast radius quantifies what a rollback cannot repair — refunds issued, bookings cancelled, messages sent — and pushes tracking to the tool boundary.

Industry relevance: The three-way outcome (admit / reject / refuse) is directly implementable as an admission controller, and the paper's claim that a σ measured before a model or settings change counts as unmeasured is an operational rule teams can adopt immediately. The finding that failures of request-level guarantees are silent — a gateway that forwarded requests for a fixed output format without the format, forced tool calls incompatible with extended reasoning, judges that reject temperature settings, all measured 2026-08-13 with one provider through one gateway — is a concrete warning about trusting documentation.

Future Directions

• Confirming the σ→regret relationship at scale. The airline experiment was underpowered: the authors estimate roughly 220 held-out test tasks would be needed, and both airline and retail candidate pools were "field-limited" (gap-to-noise ratios of 0.62 and below 0 for retail), so a richer candidate pool or larger task set is a prerequisite.
• Calibrating σ_min per platform. The paper gives a four-step procedure (set a tolerable regret r, build a non-field-limited pool, fit regret against σ, pick the smallest σ whose upper confidence bound on regret is at most r) but leaves the actual thresholds to future work, since the crossover moved by ±0.15 with the number of averaged runs.
• Testing change types other than prompt edits end to end. The verdicts for tool, skill and weight changes follow from the definitions but are not separately tested; only prompt edits were run through the full loop.
• Bounding false admits, not just σ. Because only false admits let a worse change through, the authors suggest additionally requiring an upper confidence bound on α below some α_max, counting tasks rather than runs — the judge's 4 false admits are one task seen four times, pushing the upper bound near 0.37 rather than the 0.20 that counting runs suggests.
• Correcting the loop's overstated gain, either by promoting on a lower confidence bound (as in off-policy evaluation) or with statistical corrections for selecting the best of several candidates; the authors recommend both but have not compared them on their data.

Target Audience

Platform and infrastructure engineers building agent-improvement or auto-remediation loops; applied ML scientists who design LLM-judge gates and need to know when a judge is silently broken; evaluation and reliability engineers responsible for admission control and rollback policy; and researchers working on self-improving agents, reward gaming, and LLM-as-judge validity. It is also useful for engineering managers deciding which agent changes may ship without human review — the rule and the two acceptance tests are inexpensive and transferable even if the specific platform is not.

Authors’ abstract

Agent platforms are starting to improve their own agents: they read a deployed agent's execution traces, derive a change, have an automated verifier approve it, and release it with no human author. How much of this loop can safely run without a human? We propose two quantities a platform can measure. Verifier strength $σ$ is how well the available verifier -- often an LLM judge -- agrees with a trusted reference verifier; it limits how much a step can improve the agent. Recoverability is how cheaply a bad change can be undone: whether undoing it needs a new verifier approval, and how much of the old behavior a rollback restores. A step should run unattended only if both are adequate. On our platform this rule allows prompt, tool and skill edits to run unattended and keeps weight updates under human sign-off. Our main contribution is these two definitions and how to measure them. We built a platform that applies the rule and tested it in two ways: on the platform's own approval gate, and in controlled experiments on the $τ^2$-bench agent benchmark. On verifier strength, the platform's LLM judge sometimes gave the same score no matter what it was shown (9 of 37 settings) and reported no error, so a verifier can look healthy while approving anything; its strength must be measured, and two simple tests catch the failure. On recoverability, restoring a saved copy brought the agent back to its original behavior, but a regenerated undo did not, so only changes whose old version is kept can be reversed without a human; and while a change was deployed, it altered the outcome of 17 percent of tasks beyond normal run-to-run variation, which no undo repairs. Together these support the rule: a change can run unattended only when its old version is kept and its verifier's strength has been measured.