RDD: Retrieval-Based Demonstration Decomposer for Planner Alignment in Long-Horizon Tasks

AI summary

Overview

• Research area: Robotics — hierarchical vision-language-action (VLA) models for long-horizon manipulation, specifically the automatic temporal segmentation (decomposition) of demonstration videos used to finetune VLM-based high-level planners.
• Technical level: Advanced. The paper combines VLA/robotics systems engineering with a formal optimal-partitioning formulation, a dynamic programming solver, complexity analysis, and approximate nearest neighbor search.
• Scope: The paper proposes a training-free method that decomposes long-horizon robot demonstrations into sub-tasks chosen to visually match the training data of the low-level visuomotor policy, and evaluates it on RLBench, a real-world AgiBotWorld-Alpha task, and an out-of-distribution (OOD) benchmark.

Authors: Mingxuan Yan, Yuping Wang, Zechun Liu, Jiachen Li (University of California, Riverside; University of Michigan; Meta AI). arXiv:2510.14968v1 [cs.RO], 16 Oct 2025.

What This Paper Is About

Long-horizon robot tasks are handled by a hierarchy: a VLM "planner" breaks the task into sub-task instructions, and a low-level visuomotor policy executes them. To finetune that planner, demonstration videos must be split into sub-task intervals, which today requires human annotation or hand-designed heuristic rules. The problem is that heuristically generated sub-tasks can deviate significantly from what the low-level policy was actually trained on, hurting task performance. RDD addresses this by automatically decomposing demonstrations into sub-tasks that are visually similar to the sub-tasks already present in the visuomotor policy's training set.

Key Contributions

1. First method to coordinate the high-level planner with the low-level visuomotor policy in a hierarchical VLA framework by generating the planner's finetuning dataset so that it is aligned with what the visuomotor policy can handle.
2. RDD, a training-free retrieval-based decomposition framework that models demonstration decomposition as an optimal partitioning problem, solved efficiently by a dynamic programming solver, with the solver's complexity and properties analyzed theoretically (brute force is O(2^(N-1)); the DP solver is O(N^2); with bounded interval length it reduces further, and O(N) when the maximum interval length L_max is bounded).
3. A concrete interval scoring and similarity design, including a combined visual-plus-temporal similarity measure with weight α, and an alternative scoring function for OOD scenarios that mixes retrieval against the policy's training set with a general change-point heuristic term (based on UVD) weighted by β.
4. Evaluation on simulation and real-world benchmarks, including 13 RLBench tasks, a real-world AgiBotWorld-Alpha "supermarket" task, and an OOD setting, showing that RDD outperforms the state-of-the-art heuristic decomposer and is robust across settings.

Main Findings

• Near-oracle simulation performance: On 13 RLBench tasks (10 random seeds), RDD reaches 74.9 ± 6.9 average success rate (avg. rank 2.2 ± 0.9), versus the Expert heuristic decomposer's 75.1 ± 4.7 (rank 2.2 ± 1.0) — a gap of only 0.2%.
• RDD beats UVD and Uniform: UVD scores 71.4 ± 5.1 (rank 3.0 ± 1.3) and Uniform 71.3 ± 5.4 (rank 3.1 ± 1.2). Notably, UVD performs similarly to naive uniform splitting, which the authors interpret as evidence that visual-feature change points are not always aligned with the visuomotor policy's training samples.
• Planner finetuning is necessary and beneficial: The planner without finetuning ("w/o Finetune") scores 52.6 ± 8.2 (rank 4.5 ± 1.2); all finetuning-based methods improve over the vanilla Llama planner by more than 35%.
• Robust to the choice of visual encoder: With LIV the average success is 81.1 ± 0.9 (rank 3.7 ± 1.6), R3M 80.0 ± 3.5, VIP 75.3 ± 3.4, VC-1 75.5 ± 3.1, CLIP 78.2 ± 2.1, DINOv2 78.4 ± 2.4, and ResNet 81.1 ± 2.5 (rank 3.4 ± 1.5). VC-1 and VIP — the only encoders with no form of language integration during training — perform worst, which the authors read as evidence for the importance of language integration in VLA perception.
• Temporal weighting matters, with a clear sweet spot: With α = 0 (no temporal alignment) the average success is 75.0 ± 2.5; α = 0.5 gives 75.7 ± 2.4; α = 1 gives 81.1 ± 0.9 (rank 2.3 ± 1.4); α = 2 gives 76.2 ± 3.0 (rank 2.2 ± 0.8). The authors note α = 0 confuses sub-tasks whose beginning and ending frames are similar (e.g., reciprocating motion), while over-weighting temporal similarity ignores semantic relationships.
• Data efficient: With one demonstration per task RDD scores 77.9 ± 4.5 (rank 2.0 ± 0.9); with three it scores 81.1 ± 0.9 (rank 1.6 ± 0.6). UVD with three demonstrations scores 75.6 ± 1.8 (rank 2.4 ± 0.6).
• Strong on real-world data: On the AgiBotWorld-Alpha real-world benchmark ("supermarket" task), using 152 demos to build the database and 37 for testing, RDD achieves 0.706 IoU versus UVD's 0.506.
• Improves under OOD sub-tasks: On the OOD evaluation described as human-operated demonstrations from RoboCerebra (560 demos for the database, 140 for testing; labeled LIBERO in Table 5), UVD achieves 0.598 and RDD achieves 0.624 (β = 0.25), 0.630 (β = 0.10), and 0.614 (β = 0.05). The authors report RDD is less sensitive to β, with the choice depending on application.
• Linear-time scaling in practice: Running Algorithm 1 on one CPU core of an AMD EPYC 9254, runtime grows linearly with the number of frames when the maximum interval length L_max is known, matching Corollary 3.2; the algorithm also supports parallel scoring.
• Finetuning on the target task remains necessary: A planner finetuned before deployment on "Close Jar", "Insert Peg", and "Install Bulb" (learning visual features but not the new task decompositions) scores 77.9 ± 4.3 (rank 1.6 ± 0.5) on the remaining tasks versus RDD's 79.6 ± 7.2 (rank 1.4 ± 0.5), averaged over 10 seeds.
• RDD outperforms a strong general VLM decomposer: The paper reports that RDD outperforms a Gemini-2.5-pro-based decomposer using a keyframe-index prompt, but the numeric values of that comparison table are not present in the provided text (the table is truncated after the method name).
• Qualitative behavior: Visualizations on real-world and simulation benchmarks show RDD being robust to task-irrelevant interference and to subtle arm movements where keyframe localization is hard, while UVD fails to locate keyframes precisely and its sub-tasks largely deviate from expert sub-tasks.

Methodology in Plain English

The setup is a two-level robot system. A VLM planner looks at the scene and the overall goal (for example, "put the cube in the drawer") and issues short instructions like "first, pick up the cube." A low-level visuomotor policy then turns those instructions into actions. The planner needs to be finetuned for each new task, and that requires dividing demonstration videos into sub-task intervals with language labels.

RDD's idea is to choose the split points so that each resulting interval "looks like" something the low-level policy has already seen. The authors take a pre-trained visual encoder and embed each frame into a compact vector. An interval is represented by the concatenation of its first and last frame embeddings — the ending frame carries goal information, and the starting frame provides context. All the sub-task intervals in the visuomotor policy's training set are placed into a vector database, and each candidate interval is matched to its approximate nearest neighbor in that database. Similarity combines (a) the distance between the interval and its retrieved neighbor in embedding space and (b) how close their durations are, balanced by the weight α. Angular distance is used as the metric.

Choosing the best way to cut a demonstration into intervals is framed as an optimal partitioning problem: maximize the total score, where the score of a partition is the sum of the individual interval scores (an additive objective). Brute force would require O(2^(N-1)) evaluations for an N-frame demonstration, so the authors use a dynamic programming solver exploiting the principle of optimality, requiring O(N^2) evaluations of the interval score. Because robot sub-tasks are typically short (tens of seconds), the algorithm can ignore overly long intervals, and if interval length is bounded between L_min and L_max the cost becomes O((L_max − L_min) · max(L_max − L_min, N − L_max)) — linear O(N) when L_max is bounded. The authors assume only L_min = 2 for sanity and make no assumption on L_max.

For retrieval, they use Annoy (a random-projection-tree approximate nearest neighbor method) with 10 trees, noting RDD also works with GPU-accelerated libraries like FAISS.

In the OOD setting, where genuinely new sub-tasks the policy has never learned may appear, RDD switches to a different similarity: the distance is computed only on the ending frame (since OOD sub-task durations are unpredictable), and a second term weighted by β rewards intervals that agree with a generic change-point heuristic (UVD) for identifying "general" sub-tasks. A novelty indicator Δ — the average interval score of the optimal partition — flags when new sub-tasks are present; in that case the new sub-tasks are used to finetune both the visuomotor policy and the planner.

Experimental setup: the authors build on the RACER hierarchical VLA framework, using RVT as the low-level policy and llama3-llava-next-8B as the planner's pre-trained base. The planner is finetuned for two epochs on the demonstration set using LoRA (rank 128, scaling factor 256), taking about 5 minutes on 4 NVIDIA 6000 Ada GPUs. Gemini-1.5-flash generates sub-task language instructions. The visuomotor training data comes from RLBench: the original 1908 teleoperated demonstrations (RACER's augmented version, which contains 10,159 demonstrations, is not used, to exclude interference), split into 12,700 sub-task intervals by a task-specific heuristic based on gripper state, arm stopping, and demonstration end. The demonstration finetuning set contains three demonstrations per task, and the evaluation covers 13 RLBench tasks where the low-level policy has decent performance (over 35% success rate with the expert planner); performance on all 18 tasks appears in the appendix. Baselines are the Expert heuristic decomposer, UVD, Uniform (10 equal partitions), and a planner without finetuning.

Why This Matters

• Research impact: The paper reframes planner finetuning data generation as an alignment problem between the high-level planner and the low-level policy, rather than a generic video-segmentation problem. It provides a training-free, annotation-free pipeline with formal complexity guarantees, and shows that a purely perception-driven decomposer (UVD) can be no better than naive uniform splitting in a hierarchical VLA context.
• Real-world applications:
  • Household service robots performing multi-step chores such as closing jars, opening drawers, placing wine, turning taps, and sweeping into a dustpan — all tasks evaluated in this paper.
  • Retail and supermarket settings, where the real-world AgiBotWorld-Alpha "supermarket" task was used with 152 database demos and 37 test demos.
  • General manipulation with diverse objects, task goals, and arrangements, as represented by the OOD human-operated demonstration dataset (RoboCerebra) used in this work.
  • Industrial or lab assembly-style procedures, such as installing a bulb or inserting a peg, which appear among the RLBench tasks used as the pre-deployment finetuning tasks.
• Industry relevance: Training data for robot foundation models is expensive, and manual sub-task annotation is slow, subjective, and inconsistent. A decomposer that requires no human labels, runs training-free, scales linearly when sub-task lengths are bounded, and works with multiple off-the-shelf encoders lowers the labeling cost of building planner finetuning data and reduces the risk that a planner issues instructions the deployed policy cannot execute.

Future Directions

• Closing the remaining gap to expert decomposition: RDD reaches 74.9 ± 6.9 versus the Expert's 75.1 ± 4.7 on 13 RLBench tasks; the authors note variability differences across methods, and further improvement toward the upper bound remains open.
• Better handling of OOD and novel sub-tasks: The β-weighted generalization term is reported as relatively insensitive, with the best OOD IoU of 0.630 at β = 0.10; how to set this automatically or discover genuinely new sub-tasks without a borrowed change-point heuristic is unresolved.
• Scaling the retrieval pipeline: The paper notes RDD can work with GPU-accelerated ANNS libraries such as FAISS for further acceleration and provides a conceptual speed evaluation in the appendix; validating this at much larger database sizes remains a next step.
• Reducing dependence on target-task finetuning: The zero-shot transfer experiment (77.9 ± 4.3 for a planner finetuned only on three other tasks versus 79.6 ± 7.2 for RDD) shows target-task finetuning is still necessary, raising the question of how much of that finetuning can be eliminated or replaced by retrieval alone.

Target Audience

Researchers and engineers working on robot learning, hierarchical VLA systems, and long-horizon manipulation who need to build planner finetuning datasets without manual annotation. It is also relevant to practitioners deploying VLM planners over learned visuomotor policies, and to readers interested in optimal partitioning, dynamic programming, or approximate nearest neighbor retrieval applied to robotics data. Readers without background in imitation learning, VLM planners, or dynamic programming will find the mathematical formulation dense, though the core intuition — split demonstrations into pieces that look like what the low-level policy already knows — is straightforward.

Authors’ abstract

To tackle long-horizon tasks, recent hierarchical vision-language-action (VLAs) frameworks employ vision-language model (VLM)-based planners to decompose complex manipulation tasks into simpler sub-tasks that low-level visuomotor policies can easily handle. Typically, the VLM planner is finetuned to learn to decompose a target task. This finetuning requires target task demonstrations segmented into sub-tasks by either human annotation or heuristic rules. However, the heuristic subtasks can deviate significantly from the training data of the visuomotor policy, which degrades task performance. To address these issues, we propose a Retrieval-based Demonstration Decomposer (RDD) that automatically decomposes demonstrations into sub-tasks by aligning the visual features of the decomposed sub-task intervals with those from the training data of the low-level visuomotor policies. Our method outperforms the state-of-the-art sub-task decomposer on both simulation and real-world tasks, demonstrating robustness across diverse settings. Code and more results are available at rdd-neurips.github.io.