ProgressCompass: Embodied Progress Reward Models Are Lost Without the Right Context

AI summary

Overview

Research area: Embodied AI / robot learning, specifically Progress Reward Models (PRMs) that score how far a long-horizon robot task has progressed; sits at the intersection of Natural Language Processing, vision-language modeling, and robot manipulation.

Technical level: Intermediate. The reader needs basic familiarity with reward models, vision-language models, and metrics such as MAE and Spearman's ρ, but the paper's central argument is conceptual rather than mathematically dense for most of its length.

Scope (1 sentence): The paper builds a hand-annotated benchmark (ContextProgress-Bench) isolating three forms of context-dependent progress estimation, diagnoses five existing PRMs on it, proves a theoretical error floor, and proposes an agentic loop (ProgressCompass) that supplies context to a frozen PRM using general-purpose VLMs.

What This Paper Is About

Progress Reward Models score how far a robot task has come at each step, but many long tasks cannot be scored from the current camera frame alone — progress depends on what happened earlier. The authors call this context-dependent progress estimation and show that current PRMs fail at it even when given the entire observation history, because history is a raw record rather than the specific interpreted fact the model needs. Their goal is to (a) measure this failure rigorously and (b) show it can be fixed without retraining the PRM, by having general-purpose VLMs produce the missing context.

Key Contributions

1. A problem formulation and benchmark. The paper defines context-dependent progress estimation and builds ContextProgress-Bench, containing 24 hand-selected manipulation tasks, 120 episodes, 552 annotated subtask intervals, and 71,708 frames, drawn from RMBench, RoboDojo, and LIBERO-Mem. The benchmark isolates three forms: State Recall, Sequence Tracking, and Recurrence Disambiguation.

2. A paired diagnosis of five PRMs. Using the same input format in both runs and varying only the context, the authors show every PRM is far off without context (MAE between 15.6 and 31.0 on a 0 to 100 scale) and improves sharply with it (MAE falls by 77% to 82%, to between 3.4 and 6.7).

3. ProgressCompass, an autonomous agentic loop. The loop keeps the PRM frozen and uses VLMs to supply context: an Orienter (context), a Verifier (completion check), and a text-only Navigator (control).

4. A theoretical account. Theorem 1 gives a context floor — a lower bound on the error of any estimator reading only the instruction and current frame — and an upper bound on ProgressCompass's error decomposed into a position term and a local term.

Main Findings

• Without the right context, every PRM is badly lost. On the same input format, MAE ranges from 15.6 to 31.0 on a 0 to 100 scale across the five models: ProgressLM (17.95 MAE, ρ 0.74), Robo-Dopamine (15.59, ρ 0.74), RoboMeter (25.42, ρ 0.53), TOPReward (30.95, ρ 0.56), and VLAC (24.43, ρ 0.72).

• With the correct context, error collapses. Oracle-context MAE falls by 77% to 82%, to 4.18, 3.43, 4.91, 6.70, and 4.50 respectively, with rank agreement reaching 0.90 to 0.99. The gain holds on 93.3% of paired intervals rather than on a few easy episodes.

• Having the history is not having the context. PRMs that read or retrieve from the entire history still get lost, because the history must be interpreted for the instruction and the current moment.

• Self-chained context helps, but far less. Anchoring the context on the model's own previous endpoint still improves every PRM, but to a much smaller degree: −7% (ProgressLM), −70% (Robo-Dopamine), −55% (RoboMeter), −70% (TOPReward), −30% (VLAC).

• Three characteristic failure modes. State amnesia (a state change leaves the frame and the curve falls back instead of accumulating), phase drift (a step is read as a later one than it is), and occurrence confusion (near-identical frames get near-identical values — eight button presses give one value instead of a staircase).

• ProgressCompass more than halves the backbone's error. Wrapping the frozen RoboMeter-4B drops MAE from 25.4 to 9.3 and lifts rank agreement from 0.53 to 0.93 — a 63% error cut, a 76% rank-agreement rise, and 78% of the oracle-context gap closed. It is the best method on every context form, ahead of every frozen model and of R²VLM.

• Robust to instruction–execution mismatch. Across 15 held-out negatives: on Early stop, TOPReward deviates by 50.3 and RoboMeter by 32.7 against 7.3 for ProgressCompass; on Extra steps, ProgressCompass deviates by 15.8 against 24.0 to 40.2 and alone reaches 100; on Mismatch, ProgressCompass never exceeds zero, where ProgressLM, Robo-Dopamine, and TOPReward deviate by 25.4, 49.1, and 32.5.

• The expected transition matters. Removing the expected transition at the Verifier doubles subtask-end error from 8.6 to 17.2; removing it at the Orienter as well raises it to 18.8. Progress MAE rises from 9.3 to 17.5 and 18.3, while stalled, unverified subtasks rise from 7 to 40 and 45 episodes — verification changes, not planning.

• The loop parallelizes. Running up to five episodes at once with a dependency-aware scheduler cuts per-episode time from 114.7 s to 39.5 s (a 65.6% wall-clock reduction) with MAE and ρ unchanged within paired bootstrap intervals.

• A theoretical floor and bound. Theorem 1 shows any current-frame estimator pays at least δ_S + δ_Q + δ_R (one floor per context form), while ProgressCompass's error is bounded by η^PC/K (position, governed by the Verifier) plus λ/K (local, governed by the PRM). The comparison ProgressCompass better than all current-frame estimators is stated as a condition, not a guarantee.

Methodology in Plain English

The authors first hand-select robot-manipulation tasks from three existing trajectory datasets where the current frame genuinely cannot reveal progress, then hand-annotate the subtask boundaries and the specific context each episode needs at each interval.

They then run a paired diagnosis: each of five PRMs is evaluated twice on all 552 annotated subtask intervals using the identical input format. In one run the model gets only the episode instruction; in the other, the instruction is replaced by the active subtask instruction, which embeds the hand-written context. Because only the context differs, any change in MAE or Spearman's ρ is attributable to context alone. They also test a "self-chained" variant, where the model's own previously predicted endpoint, rather than the true one, anchors the current estimate.

To build ProgressCompass, they split the labor. A PRM scores a clip once it knows which step is underway, so RoboMeter-4B stays frozen as the scorer. A VLM (Qwen3.5-27B) is the Orienter, reading frames and naming the current step and its expected transition. A second VLM (Qwen3.5-9B) is the Verifier, ruling on whether that step is done — deliberately separate from the Orienter, since the component that says what should happen should not also rule on whether it did. A third, text-only Qwen3.5-9B acts as Navigator, running the loop and carrying the plan and confirmed outcomes forward. Decoding is at temperature 0. The PRM only sees frames since the current step began.

Why This Matters

Impact on research: The paper reframes a common assumption — that progress estimation fails because PRMs are weak models — as a context problem. It supplies a benchmark that isolates the failure, a theoretical floor proving no amount of model capacity fixes it, and a method showing existing frozen models can be repaired at inference time. This suggests a general recipe for long-horizon embodied agents: give the scorer the right fact, not the whole history.

Real-world applications (implied by the paper's framing):

• Dense reward signals for robot policy training in settings where the environment supplies no reward.
• Progress monitors that detect when a robot execution has stalled, so a supervisor can intervene.
• Verifiers that judge whether a step is actually finished, for step-level task success checking.
• Guardrails that catch when an execution does less than asked (Early stop), more than asked (Extra steps), or something unrelated (Mismatch) — the paper's 15 held-out negatives target exactly these.

Industry relevance: Because the PRM stays frozen and no training is required, ProgressCompass is an inference-time wrapper that could be layered onto existing deployed progress models. The paper reports the added components are low cost and that parallelization cuts wall-clock time by 65.6%, which matters for any production robot loop with latency budgets.

Future Directions

• Removing the fixed-step assumption. The theory assumes K ordered steps per instruction; the paper does not report results for open-ended or unsegmented tasks, where the plan itself must be discovered.
• Replacing hand annotation entirely. The benchmark's context and subtask boundaries were written by hand, and the diagnosis deliberately sets every boundary itself. How well ProgressCompass would scale where no annotation exists for validation is an open question.
• Reducing dependence on the verifier's false-accept behavior. Theorem 1 charges position error to the Verifier and notes early completion passes through with probability at most β; tightening β — or measuring it directly — is a natural next step.
• Deployment on real hardware and broader task suites. The benchmark draws on three existing datasets and 120 episodes; extension to physical robots and to domains beyond tabletop manipulation is not reported.

Target Audience

Robotics and embodied-AI researchers working on reward modeling and long-horizon task execution; benchmark designers interested in controlled evaluation of context dependence; NLP and LLM-agent researchers studying context management and agentic loops; and industry teams deploying progress monitors or dense-reward components in robot systems.

Authors’ abstract

Embodied agents now take on ever longer tasks. For long tasks, knowing only whether a task finally succeeds or fails says little; the steps along the way matter. Progress Reward Models (PRMs) score how far a task has come at every step, and serve as dense rewards, verifiers and monitors. Yet in long tasks the current frame alone often cannot tell how far the task has come, because progress depends on what happened before. We call this problem context-dependent progress estimation. Existing benchmarks on progress estimation mostly focus on short tasks whose progress can be read from the current observation, and whether PRMs can estimate progress when context is needed remains underexplored. We therefore build ContextProgress-Bench, with 24 manipulation tasks for 120 episodes. The benchmark covers three settings: (i) State Recall, where information needed for progress appeared earlier but is not in the current frame; (ii) Sequence Tracking, where steps follow a fixed order, so progress requires knowing which steps are done and which comes next; and (iii) Recurrence Disambiguation, where look-alike frames sit at very different progress. We then run a paired diagnosis: each PRM keeps the same input format in both runs, and in one run its instruction integrates the right context. Even PRMs that read the entire history get lost in estimating progress, yet with the right context the same five models cut their progress error by 77-82%. Embodied PRMs are thus not incapable of progress estimation, but lost without the right context. We therefore propose ProgressCompass, an autonomous agentic loop that reorients an existing PRM and uses current general-purpose VLMs to supply the context the PRM needs. Wrapped in the loop, the same frozen PRM cuts its progress error by 63% and raises its rank agreement by 76%. With such a compass, PRMs estimate progress far better on longer, more complex tasks.