Reuse Before You Retrieve: Diagnosing Headroom and Complementarity for Test-Time Augmentation of Embodied Multimodal Policies

AI summary

Overview

Research area: Embodied AI and robot learning, specifically test-time augmentation of frozen vision-language-action (VLA) policies.

Technical level: Intermediate. The paper assumes familiarity with VLA models, stochastic policy sampling, pass@N metrics, demonstration retrieval, and flow-matching versus autoregressive action generation.

Scope: A diagnostic study that measures two properties of frozen VLA policies — recoverable headroom and retrieval complementarity — to decide whether a deployed policy benefits more from retrying and selecting among its own stochastic rollouts or from injecting external demonstrations, evaluated across LIBERO, SimplerEnv-Bridge, LIBERO-Occ, and OpenVLA.

What This Paper Is About

Frozen VLA policies can be improved at test time either by sampling multiple rollouts and selecting among them, or by retrieving external demonstrations to steer action generation, but neither intervention helps consistently and there is little guidance on which one a given policy actually needs. The authors ask whether two measurable quantities can distinguish these opportunities: how much better behavior already exists inside the policy's own stochastic rollouts (recoverable headroom), and how far the policy's sampled actions sit from the demonstrated action prior (retrieval complementarity). They then test a training-free episode-level retry selector and an event-schema retrieval method across several frozen VLA backbones to see whether these diagnostics predict which intervention works.

Key Contributions

1. Introduces recoverable headroom and retrieval complementarity as measurable diagnostics that separate test-time opportunities based on reusing a frozen policy's existing behavior from those that may require an external action prior.
2. Presents a training-free episode-level retry selector that improves multiple VLA backbones under retryable execution, with gains that closely track each policy's available recoverable headroom.
3. Shows through retrieval experiments that the tested external demonstration prior consistently helps the backbone with the largest measured action-prior gap, and that retrieval and selection provide additional gains when combined.
4. Extends the analysis to a different robot and simulator, degraded observations, and an autoregressive VLA, revealing when available headroom transfers and when the ranking signal becomes the limiting factor.

Main Findings

• Selection improves every tested configuration. The episode-level selector improved all 12 LIBERO backbone × suite cells, with average gains ranging from +7.8 to +21.0 success-rate points: π0 improved from 64.2 to 85.2 (+21.0), SmolVLA from 65.6 to 85.5 (+19.9), and π0.5 from 88.7 to 96.5 (+7.8).

• Retrieval is backbone-specific, not driven by baseline capability. Retrieval improved π0 across all four LIBERO suites (+4.5 ± 1.5 Spatial, +9.8 ± 2.5 Object, +5.5 ± 0.8 Goal, +6.0 ± 2.3 LIBERO-10), but produced small and inconsistent changes for SmolVLA (+0.5, −0.2, +1.6, −0.9) and π0.5 (−1.1, +0.6, +2.6, −0.1). π0 and SmolVLA had nearly matched mean pass@1 (64.2 vs. 65.6, a difference of 1.4 points) yet only π0 benefited consistently.

• Demonstration-fit error identifies the retrieval opportunity. Under a shared action representation and fixed sampling protocol, DFE was 0.042 for π0, 0.021 for π0.5, and 0.003 for SmolVLA over 400 demonstration states. A separate scale-independent check found the closest sampled first action correlated with the demonstrated first action at 0.91, 0.96, and 0.98 for π0, π0.5, and SmolVLA respectively.

• Selector gain tracks recoverable headroom. Across the 12 LIBERO backbone × suite cells, the capture ratio gain/H₃ had mean 0.90, standard deviation 0.11, and range [0.61, 1.00]. Selector gain correlated with recoverable headroom at r = 0.954 with bootstrap 95% CI [0.89, 0.99] and Spearman ρ = 0.958.

• The headroom relation survives disjoint evaluation. When headroom was measured on one episode split and selector gain on another with no shared episodes, the correlation across 50 random splits remained r = 0.91 ± 0.04. Within-backbone correlations across the four suites were 0.81, 0.95, and 0.79, and leave-one-backbone-out values ranged from 0.86 to 0.96.

• Gains come from selection, not from sampling alone. Randomly retaining one of three rollouts stayed near pass@1 (π0: 64.5 vs. 64.2; SmolVLA: 65.3 vs. 65.6; π0.5: 89.0 vs. 88.7), while manifold selection reached 85.2, 85.5, and 96.5 against oracle pass@3 ceilings of 87.8, 87.0, and 98.3. Increasing N from 2 to 3 also raised performance (π0: 79.2 to 85.2; SmolVLA: 81.0 to 85.5; π0.5: 94.8 to 96.5).

• The ranking score has strong aggregate association with success. Pooling rollout-level manifold scores across all LIBERO backbones and suites gave an AUC of 0.96 over 17,964 valid rollouts; of the 18,000 LIBERO rollouts, 36 lacked a valid manifold score and were excluded. The authors treat this as complementary evidence rather than a direct measure of within-episode ranking.

• Headroom does not predict retrieval gain. The correlation between recoverable headroom and retrieval gain was r = 0.32, and retrieval was near zero in eight of the twelve backbone × suite cells despite substantial selector headroom in several.

• Retrieval and selection compose. For π0, combining retrieval-warm-started rollouts with the selector reached a mean gain of +23.3, versus +21.0 from selection alone and +6.4 from retrieval alone, an additional +2.4 points on average. On LIBERO-10, retrieval raised the pass@3 ceiling from 64.4 to 71.0, and the combined method improved by a further +6.2 over selection alone; on Goal, where selection already reached 97.6%, the combination gave −0.8.

• The selector transfers across robot and simulator. On SimplerEnv-Bridge with π0-Bridge and WidowX, the selector improved overall success by +12.5 points (37.5 to 50.0) against an oracle headroom of +18.1, recovering 69% of the available opportunity, while retrieval changed overall performance by −0.9 points (carrot −4.2, spoon +1.4, stack −0.0).

• Available headroom and realized selection can diverge. On the SimplerEnv coke-can task, stochastic OpenVLA sampling alone left success at 0.40 (+0.00), per-step re-ranking (N = 4) decreased it to 0.36 (−0.04), and episode-level selection (N = 3) reached 0.44 (+0.04) against an oracle pass@3 of 0.72. The authors stress the large gap between realized selection and the oracle, not the noisy +0.04 gain.

• Occlusion changes the available opportunity. Under LIBERO-Occ, π0.5's selector gain rose from +7.8 under clean conditions to +11.2 under occlusion, approaching π0's +11.9, while retrieval gains stayed weak (+4.3 for π0, +0.8 for π0.5). π0.5's occluded mean pass@1 was 68.0 (versus 59.5 for π0).

Methodology in Plain English

The authors start by defining a diagnostic rather than a method. For any frozen policy, they run it repeatedly from the same starting state and record whether each rollout succeeds. The difference between pass@1 (one attempt) and pass@N (best of N attempts) is called recoverable headroom — it is the ceiling for any selector that picks among those N candidates. For the retrieval side, they feed the policy observations taken from successful demonstrations and sample K = 3 action chunks, measuring the distance from the demonstrated action to the closest sampled one. This demonstration-fit error (DFE) indicates whether the policy already produces the behavior a retrieved demonstration would supply.

Their selector is deliberately simple and training-free. From the same initial state, they run the frozen policy N = 3 times with different seeds. Each rollout is scored by comparing every observation frame against a library of successful demonstration frames using a frozen encoder that concatenates DINOv2 and SigLIP features into a 1,536-dimensional ℓ2-normalized vector, and averaging the best cosine similarity over the trajectory. The highest-scoring rollout is kept. No success labels or trained scoring components are used.

For retrieval, they build an event-schema method that marks sub-skill boundaries in demonstrations (episode starts, gripper open/close transitions, and sharp motion-direction changes), matches the current event using a weighted combination of standardized image-embedding, predicted-action, and proprioceptive distances, and then retrieves the action chunk for the next event in the matched demonstration. For flow-matching policies, the retrieved chunk warm-starts the sampler as a blend with the policy's own noise via α = 0.3.

The evaluation covers three flow-matching LIBERO backbones (π0, SmolVLA, π0.5) on the four LIBERO suites with n = 500 × 3 rollouts each using the Franka robot and 50 demonstrations per task; three SimplerEnv-Bridge tasks (carrot, spoon, stack) with WidowX and n = 24 × 3 rollouts using task-specific libraries of 510 frames (Bridge); OpenVLA-7B on the SimplerEnv coke-can task with 25 initial states, three sampled rollouts per state, and a library of 1,360 frames; and LIBERO-Occ with n = 250 × 2 rollouts and N = 2. Episodes run up to 400 environment steps with 50-step action chunks. The retrieval configuration and α were selected once on a held-out split and frozen across all backbones, so the backbone-specific retrieval pattern reflects a controlled comparison rather than per-backbone tuning. Preliminary runs with n = 100 episodes occasionally showed changes of up to roughly four success-rate points that did not persist under the full protocol, which is why the larger protocols were used.

Why This Matters

The paper reframes test-time augmentation as a diagnosis problem: before applying a retry-and-select scheme or a retrieval module, a practitioner can measure whether the frozen policy already contains better behavior in its stochastic rollouts and whether it already reproduces the demonstrated action prior. This is a shift from designing better selectors or better retrieval representations toward deciding which intervention a deployment actually requires, and it supplies evidence that baseline success rate alone is insufficient for that decision.

Real-world applications:

• Warehouse or factory robot arms running manipulation policies where a failed attempt has no irreversible consequence, making repeated or parallel execution and retry selection practical.
• Deployed service or household robots with degraded camera views or occlusions, where the paper shows the retry selector remains effective and the amount of recoverable headroom grows.
• Robot fleet operators deciding whether to invest compute in multi-rollout inference or in maintaining a demonstration library and retrieval pipeline tailored to a specific policy checkpoint.
• Cross-platform deployments, such as transferring a policy trained in one simulator or robot embodiment to another, where the selector transferred to WidowX in SimplerEnv while retrieval did not improve overall performance.

Industry relevance: The selector is training-free and requires only nearest-neighbor comparisons over frozen embeddings, so its additional cost is the straightforward N× overhead of running N rollouts — attractive for systems that already provision parallel inference or can reset to a common initial state. The DFE probe offers a cheap pre-deployment check for whether a retrieval subsystem will pay off for a given backbone, which matters when retrieval infrastructure is expensive to build and maintain.

Future Directions

• Isolating the cause of weak headroom capture on OpenVLA. The paper notes that the OpenVLA evaluation differs from LIBERO in several respects, including available observation views, and does not attribute the weaker ranking performance to any single factor.
• Developing stronger trajectory-ranking signals so that existing headroom can be recovered when it is present but not identifiable, since the gap between oracle pass@3 (0.72) and realized OpenVLA selection (0.44) is large.
• Calibrating demonstration-fit error across policy families and action spaces. DFE currently depends on the action representation and sampling protocol and is interpreted comparatively within a shared action space rather than as a universal predictor.
• Extending the analysis beyond simulation and beyond the tested backbones and retrieval mechanisms, as all experiments reported here are conducted in simulation.
• Establishing an equal-budget comparison between selection and retrieval, since the two incur different forms of test-time cost and this work diagnoses their distinct opportunities rather than normalizing cost.

Target Audience

Researchers and engineers working on embodied AI, robot manipulation, and VLA policy deployment who are considering test-time interventions without weight updates. It is also relevant to practitioners building inference infrastructure for robots, since the decision between multi-rollout selection and demonstration retrieval has direct compute and data-collection implications, and to method developers working on test-time scaling and retrieval-augmented policies who want a diagnostic vocabulary for evaluating their interventions.

Authors’ abstract

Frozen vision-language-action (VLA) policies are increasingly improved at test time by sampling additional policy behaviors or introducing external demonstrations. Yet there is little guidance for deciding which intervention a deployed policy actually needs. Additional sampling is useful only when better behavior already exists within the policy's stochastic rollouts and can be identified, whereas retrieval is most useful when the relevant action prior is not reliably represented by the policy. We study this decision through two measurable factors, recoverable headroom and retrieval complementarity, which characterize how much useful behavior is already available to recover and whether an external action prior fills a measurable gap. We evaluate an episode-level retry selector under retryable or parallel execution, together with retrieval across multiple frozen VLA policies and environments. The selector consistently recovers substantial latent capability across all tested VLA backbones on LIBERO, with gains of up to 21.0 success-rate points that closely track recoverable headroom. It also transfers to a different robot and simulator and remains effective under degraded observations, while experiments with autoregressive OpenVLA illustrate the distinction between available headroom and the ability to rank candidate rollouts. Retrieval behaves differently, improving the policy with the largest measured action-prior gap and providing further gains when combined with selection. Together, these results provide an empirical basis for characterizing test-time augmentation opportunities by separating capability that can be recovered from the frozen policy from behavioral priors that may need to be introduced externally.