Post-Audit Mirage: Identifiability Limits of Offline Agent Verification

AI summary

Overview

Research area: AI agent safety and evaluation — specifically, whether offline audits/verifiers can establish that an agent update is safe to deploy.

Technical level: Intermediate. The core argument is intuitive, but the paper leans on statistical identifiability, confidence sequences, and anytime-valid inference, so some background in hypothesis testing helps.

One-sentence scope: The paper proves that when a safe and a harmful deployment produce byte-identical offline audit records, no verifier can distinguish them, then builds an executable benchmark of such matched world pairs and a fail-closed statistical monitor that uses fresh post-deployment outcomes to recover a narrow, declared group-harm decision.

What This Paper Is About

Offline tests of an agent update only see a fixed audit record, but deployment can change who reaches which route, how capacity is shared, or when a rule actually fires — producing harm that never appeared in the record. The authors formalize a "post-audit mirage": a single audit record can be compatible with both a safe and a harmful deployment, so the verifier receives literally identical input in both worlds yet the correct action is opposite. The goal is to make that indistinguishability boundary executable, quantify the resulting error, and test whether collecting fresh group-labelled harm outcomes after the update is frozen can restore a decision.

Key Contributions

1. An executable indistinguishability boundary. The authors apply standard observational-equivalence and two-point indistinguishability arguments to an agent-audit interface. The Audit-View Non-Identifiability Theorem states that if a safe world and a harmful world induce the same distribution over the public record O, then every offline rule A(O, U) — where U is private randomness independent of O with the same distribution in both worlds — has the same decision distribution in both worlds, and with abstention probability r, max{P_harm(A = deploy), P_safe(A = hold)} ≥ (1 − r)/2. They operationalize this with byte-identical public records and isolated scorer-only truth.

2. The Post-Audit Mirage benchmark. An exact matched-world lifecycle benchmark across three environments — AuthGate (route access changed by group), ConstraintPlan (incumbent schedule replaced), and BatchTriage (shared-server policy changed) — built as safe-harmful pairs that share one serialized public lifecycle object while the evaluator receives a separate realized environment object. Construction fails unless public rounds are identical, protected answers are opposite, and every public-only method returns the same paired decision sequence.

3. IdentifiedRangeMonitor. A fail-closed monitoring method that consumes fresh group-labelled binary harm outcomes after the update is fixed and returns one of four statuses: deploy, hold, cannot determine, or unsupported. It deploys only when every required group's upper bound is at or below the threshold, holds when any lower bound is above it, and otherwise remains undecided; a biased or shifted declared collection design returns unsupported.

4. An orthogonal evaluation question for agent benchmarks. Where existing agent benchmarks and systems (SGM, PACE, SEA, SEAL, SEA-Eval) measure task success, budget evidence across edits, or benchmark trajectories, this paper asks whether the verifier's permitted observations identify the deployment property at all.

Main Findings

• Identical audits force identical decisions. Across 272 pairs and 50 rounds per pair, all 13,600 public inputs match byte for byte, protected truth reverses, and all nine offline procedures return identical paired sequences when given the same random draws. The authors are explicit that this verifies observation equivalence without scorer leakage, not that the procedures are generally equivalent or poor.

• The theorem gives a hard floor on error. If a rule abstains 20% of the time, at least one matched world has error of at least 40%. For a deterministic non-abstaining rule, the shared observation forces either deploy in both worlds or hold in both worlds, so one decision is wrong with probability one.

• Fresh outcomes can restore the distinction on a fixed panel. In the fixed-panel restoration study, each lifecycle is sampled with 20 streams per world and aggregated with a prespecified 16-of-20 rule. At 20,000 attempted observations, all 272 safe and 272 harmful lifecycles meet the descriptive correctness rule, with 0/272 harmful lifecycles incorrectly deployed as safe. The authors stress this is a descriptive result on a fixed synthetic panel, not a population error-rate estimate, and that zero observed false-safe lifecycles is not zero risk.

• Recovery is slow and undecided in the middle. On the fixed panel, at 2,500 observations the safe world deploys 62.5% of the time and is undecided 37.5%; at 5,000 and beyond it deploys 100%. The harmful world holds 63.2% and is undecided 36.8% at 2,500 and reaches 100% hold at 5,000.

• Most controlled difficulty settings never qualify. Sweeping gaps of 0.025, 0.05, and 0.10 around the 15% threshold against rare-group prevalence of 0.02, 0.10, and 0.50 and retention of 0.5 or 1.0, with 500 independent streams per world and cell, only four settings qualify with the primary Hoeffding monitor within the inspected range. Smaller margins, rarer groups, and missing observations keep the monitor undecided longer.

• On the full controlled grid, most cells stay undecided. At 20,000 attempted observations the controlled grid shows 30.0% deploy and 70.0% cannot determine in the safe world, and 30.1% hold and 69.9% cannot determine in the harmful world, with 0.0% deploy in the harmful world and 0.0% hold in the safe world.

• Method choice changes coverage under the same data. Of the 18 prespecified cells, the Hoeffding confidence sequence qualifies in 4/18, the beta-binomial confidence sequence in 9/18, and the pointwise Wilson interval in 12/18 — but Wilson is fixed-sample only and not valid for optional stopping, so its onsets are descriptive.

• On the fixed panel, all three interval methods agree at the final look. Hoeffding, Wilson, and beta-binomial all reach 100.0% deploy in the safe world and 100.0% hold in the harmful world at 20,000 observations, with unsupported at 0.0% throughout.

• The guarantee is narrow by construction. The monitor certifies only declared group harm — not feasibility, utility, undeclared groups, or complete deployment safety — and assumes independent binary harm outcomes with a stable rate per required group, a group list fixed in advance, missingness unrelated to harm, and no change in the target outcome distribution.

Methodology in Plain English

The authors first define the audit setting precisely: there is a public record O that the verifier is allowed to see, plus some deployment truth Z that only an isolated scorer sees. A verifier is any rule that maps O and its own private randomness to deploy, hold, or abstain. They then construct world pairs where the public record is engineered to be byte-identical while the protected truth points in opposite directions, and prove that any such verifier must behave the same way in both — leaving it with no option but to be wrong somewhere or abstain often.

To make that boundary concrete rather than hypothetical, they build a generator that produces safe-harmful lifecycle pairs for three environments, each running 50 update rounds. A paired constructor asserts that public rounds match, protected answers are opposite, and every public-only procedure receives coupled inputs and randomness before it will write a witness. They then run nine public-only diagnostic rules — always hold, greedy, fixed threshold, shrinking-budget spending, ADDIS-style spending, online closed e-testing, per-update reset, reused holdout, and transferred cross-update evidence — to confirm none of them can break the tie when only the audit is available.

Finally, they test whether new information helps. The IdentifiedRangeMonitor observes binary harm outcomes for a pre-declared roster of G groups after the update is frozen. For group g at its n-th retained outcome it spends α_{g,n} = α/(G·n(n+1)) and builds a Hoeffding radius r_{g,n} = sqrt(log(2/α_{g,n}) / (2n)); a union bound gives simultaneous coverage across all groups and observation counts, so the procedure is valid under optional stopping. It deploys only if every group's upper bound is safe, holds if any lower bound is harmful, and otherwise says cannot determine. The authors compare it against a pointwise Wilson score interval and an anytime-valid beta-binomial mixture confidence sequence using identical counts, and run three studies: a matched audit check, a fixed-panel restoration test with nested Monte Carlo streams, and a controlled difficulty sweep over gaps, group prevalence, and retention.

Why This Matters

Impact on research. The paper reframes verifier evaluation: the question is not only whether a verifier computes correctly, but whether the evidence it is permitted to see identifies the deployment property at all. That separates two failure modes that agent-safety work often conflates — a buggy verifier and an unanswerable question — and it gives benchmark designers a construction (matched worlds with byte-identical public records and reversed protected truth) for testing the second.

Real-world applications:

• Access-control and authorization changes. The AuthGate vignette describes a hospital testing a rule that grants a time-limited export token to approved on-call clinicians; in the safe world the gateway expires every token before the next shift, while in the harmful world a separately configured renewal worker keeps some tokens active for three steps and a later routing change sends more rare-group records through those sessions. Neither configuration appears in the pre-deployment audit.

• Scheduling and capacity planning. ConstraintPlan models replacing an incumbent schedule, where cooldowns, protected blackouts, and delayed violations are visible only after deployment.

• Shared-resource triage. BatchTriage models a shared-server policy change where co-arrival correlation, contention, backlog, and delayed group harm emerge only once the policy is live.

• Deployment governance. The monitor's four-way output gives a deployment team a way to refuse unsupported decisions and surface missing groups rather than treating a passed audit as authorization.

Industry relevance. The result argues against treating a clean offline audit as a deployment green light, and toward collecting fresh group-labelled outcomes after the update is frozen. It also warns that a declared group roster can omit affected people, and that a conservative monitor can delay beneficial updates — so the output is framed as input to human governance, not autonomous authorization.

Future Directions

• Estimating production prevalence. The benchmark constructs exact observation equivalence but explicitly does not estimate how often such equivalence occurs in real deployments, or how often monitoring succeeds there. Measuring that is an open empirical question.

• Detecting the failures the monitor cannot see. The scope section lists failures outside the guarantee: undeclared selection bias, missing-not-at-random outcomes, group omission, and target shift. The authors note that anytime-valid inference protects against optional stopping but does not repair these information failures, so methods that detect or bound them remain open.

• Improving power in hard regimes. Only four of the 18 controlled settings qualify with the Hoeffding monitor within 20,000 attempted observations, and the controlled grid is still 70.0% / 69.9% cannot determine at the final look. Closing that gap for small margins, rare groups, and partial retention — the beta-binomial sequence qualified in 9/18 cells versus 4/18 — is a direct follow-on.

• Moving beyond declared group harm. The monitor certifies only declared group harm and not feasibility, utility, or complete deployment safety. Extending certification to those properties, or to a full deployment trajectory rather than observed group outcomes, is left open.

Target Audience

AI safety and evaluation researchers who build agent benchmarks or verifiers; statisticians and ML practitioners working on sequential testing, confidence sequences, and anytime-valid inference under deployment shift; and applied teams in regulated settings — access control, scheduling, shared-resource allocation — who currently treat an offline audit as a deployment gate. Readers need enough statistical literacy to follow the identifiability argument and the distinction between pointwise and stopping-valid intervals, but the central claim is accessible without it.

Authors’ abstract

Offline tests can fail to reveal whether an agent update will remain safe after deployment. We operationalize this post-audit mirage using matched safe and harmful lifecycles that expose identical verifier-visible evidence while an isolated scorer records opposite deployment truth. Our theorem applies standard observational-equivalence and indistinguishability ideas to this executable audit boundary and states the resulting error-abstention tradeoff. The Post-Audit Mirage benchmark makes that boundary executable across authorization, scheduling, and shared-capacity environments. IdentifiedRangeMonitor uses fresh group-labelled harm outcomes to return deploy, hold, cannot determine, or unsupported. Experiments show that monitoring can recover the missing distinction in favorable settings but may remain undecided for rare groups, small effects, or missing observations. Verifier-agent evaluation must therefore test both calculation validity and whether the available evidence supports the deployment decision.