Data and Learning Where it Matters for Contact-Rich Manipulation

AI summary

Overview

Research area: Robot learning for contact-rich manipulation — automated data collection, offline deep reinforcement learning (DRL), and modular integration of planning with learned policies.

Technical level: Intermediate. Familiarity with reinforcement learning, motion planning, and manipulation terminology helps, but the paper's core argument (collect data only where it matters) is stated plainly and supported by task-level results.

Scope: The paper proposes a compositional framework that densely collects autonomous robot data only for the contact-rich "critical segment" of a task while using off-the-shelf pose estimation and motion planning for free-space motion, and demonstrates high success rates and out-of-distribution robustness across four real-world tasks.

What This Paper Is About

End-to-end policies trained on large datasets remain brittle in high-precision tasks and generalize poorly to out-of-distribution (OOD) scenarios, while data collection pipelines stay expensive. The authors argue this gap is partly a matter of what data is collected and how tasks are structured: high-precision tasks typically fail at the contact-rich critical segment (such as an insertion with tight clearance), while the rest of the task is unconstrained free-space motion. Their goal is to concentrate dense data collection and learning on that critical segment, and to solve everything else with non-robot-specific methods.

Key Contributions

1. A compositional framework that applies dense data collection and an offline-learned policy only to the critical task segment, while relying on motion planning for free-space motion.
2. An autonomous data-collection scheme that uses a mixed greedy-random exploration policy for dense data collection at the critical segment, eliminating constant dependence on a human teleoperator and on online policy updates.
3. A deployment-time integration of planning and learning that switches between them using contact events (force/torque thresholds) and learned Q-function values, achieving success rates above 94% across all four tasks.
4. Empirical evidence that failure cases of end-to-end methods concentrate at the critical segment, shown through high partial success rates but low overall success rates for imitation learning and DRL baselines.

Main Findings

• High success with little data collection: Using only 2 to 2.5 hours of wall-clock time for autonomous data collection, the method achieves 94% to 98% success across four real-world tasks and 96% on average, compared to the strongest baseline at 55%. Rollouts were 50 trials per policy.
• Per-task results (success, partial success in parentheses): Shelf stocking: Ours 98 (100), DiTFlow 38 (96), DiTFlow (Novice) 34 (80), DP 44 (100), finetuned π0.5 42 (92), MP & PE 98 (98), HIL-SERL 10 (24). Lego stacking: Ours 94 (100), DiTFlow 60 (88), DiTFlow (Novice) 22 (48), DP 44 (92), finetuned π0.5 6 (80), MP & PE 78 (100), HIL-SERL 6 (22). Fan Cover: Ours 96 (98), DiTFlow 30 (88), DP 30 (96), finetuned π0.5 20 (96), MP & PE 22 (94). Fan Cover (hard): Ours 94 (96), DiTFlow 18 (82), DP 26 (92), finetuned π0.5 12 (76), MP & PE 22 (94). Averages: Ours 96 (99), DiTFlow 37 (89), DP 36 (95), finetuned π0.5 20 (86), MP & PE 55 (97).
• End-to-end methods fail at the critical segment: Baselines show high partial success rates — indicating correct grasping and coarse placement — but low overall success rates. HIL-SERL struggled with undirected gripper actions during exploration over long task horizons, and showed high variance across training runs, which the authors note is characteristic of online policy optimization.
• Robustness to out-of-distribution scenarios: Under distractors, different object configurations, placements, and backgrounds (20 trials per policy), Ours achieved shelf stocking 90 (100), Lego stacking 85 (95), Fan Cover 90 (95), Fan Cover (hard) 95 (95). Baselines scored at or near zero in several cases (e.g., finetuned π0.5 and HIL-SERL at 0 (5) and 0 (0) for shelf stocking, and all baselines at 0 for Fan Cover).
• Lower applied forces: Maximum forces applied are reduced on average by 49% (−4.7 N) compared to baselines; the authors separate their method into the learned insertion policy and the planned push-down motion. Episodic maximum torques are reported in the appendix.
• Teleoperator skill matters: A DiTFlow policy trained on data from a novice operator achieves lower success rates. Novice demonstrations were multi-modal, had higher speed variance (CV_novice = 0.285 vs CV_expert = 0.185), and longer trajectories (27 s vs 24 s).
• Data collection efficiency: Within comparable wall-clock time, the autonomous scheme collects more data at the critical segment. For example, shelf stocking collected 300 episodes / 27k datapoints in 2 h versus 100 (22 failures) / 24k / 2.3 h for teleoperation; Lego stacking collected 500 / 30k in 2.5 h versus 100 (30) / 33k / 2.5 h. HIL-SERL required more episodes (346 for shelf stocking, 495 for Lego stacking) with many failures (133 and 221).
• Exploration rate matters (simulation ablation): Using a MuJoCo Lego setup, three seeds, and 1000 policy rollouts, policies trained with little exploration (p ≤ 0.3) achieve low success rates, as does pure exploration (p = 1). Sufficiently long rollouts with high success rates occur for 0.6 ≤ p ≤ 0.85.
• Pose estimation uncertainty scales data needs: In simulation, 400 rollouts were needed for 6 mm uncertainty and 700 rollouts for 12 mm uncertainty, corresponding to 2 h to 3.5 h of real-world data collection.
• Multi-modal sensing: Combining a wrist-mounted force/torque sensor with a single wrist camera achieves 100% success in simulation, the setup also chosen for the hardware experiments.

Methodology in Plain English

The scene is first set up with objects at fixed locations, using 3D-printed fixtures if needed, and a single demonstration is recorded (for example, by kinesthetic teaching). That demonstration is replayed until the critical segment of the task is reached — the segment defined by the uncertainty of pose estimation — and then an explorative policy takes over. With probability p = 0.8 the exploration policy samples a random action; otherwise it takes a greedy action toward the goal state. A safety filter constrains the end-effector to a sphere around a safe center so exploration stays inside the critical segment. Episodes are truncated after 10 s (150 steps), and the reward is sparse, given based on distance to the final state of the hardcoded trajectory.

The collected data trains an offline DRL policy. The algorithm builds on SAC, adds layer normalization to the critic, and uses Randomized Ensembled Double Q-Learning with an ensemble of 10 Q-functions, selecting a random subset of 2 to compute TD targets. The entropy coefficient is fixed rather than adapted, since no online learning is used. Training takes 5 epochs, roughly 1 h on an RTX5090 workstation. During deployment, the robot uses language-guided segmentation (SAM3) prompted with object names, off-the-shelf pose estimation (FoundationPose) for 6D poses, and simple waypoint following that is defined relative to the object pose and the single demonstration. The learned policy is triggered once contact occurs, detected by a force/torque threshold, and insertion success is detected from the Q-function with threshold λ_success = 0.93 R_success, plus a hysteresis maximum detector using λ_high = 0.8 R_Success and λ_low = 0.6 R_Success. Hybrid force-position control is used during the force-sensitive critical segment, applying a desired force along the insertion axis while position-controlling the remaining degrees of freedom. The policy sees a wrist-camera image cropped to the critical insertion region at 224×224 resolution, embedded with DINOv2 and projected to 16 dimensions, concatenated with state information including force/torque measurements, controller error, Cartesian velocity, and the previous action.

Why This Matters

Impact on research: The paper challenges the assumption that scaling uniformly collected end-to-end data will close the precision and generalization gap. It shows that targeted data collection plus a modular structure — learned only for the contact-rich segment, with planning elsewhere — yields higher success rates, better OOD robustness, and much lower applied contact forces than imitation learning, online DRL, and a finetuned foundation model baseline. It also shows that a vision-based success classifier can emerge from DRL training as the Q-function, avoiding the need for a separately trained reward classifier and additional demonstrations.

Real-world applications:

• Industrial assembly of molded parts on a production line, where the fan cover task originates from a real-world process that currently requires manual assembly.
• Handling sensitive or deformable components such as cardboard packaging and injection-molded PBT/PC parts, where reduced applied forces matter.
• Tight-clearance insertion tasks, such as precision alignment and press-fit operations.
• Shelf stocking and warehouse placement tasks where objects may deform or shift during interaction.

Industry relevance: The work is co-authored by researchers at TU Munich and Siemens AG, and its central claim — that robot data is expensive and should be collected only where it is difficult to solve otherwise — maps directly onto industrial deployment decisions about where to invest data-collection effort. It also reduces reliance on skilled human teleoperators, which the authors emphasize as a practical strain, and the autonomous scheme required only a few interventions when the scene was not reset correctly.

Future Directions

1. Better switching mechanisms. Q-function-based switching worked for these tasks, but the authors suggest VLM- or LLM-based task-segment switching for semantically rich tasks as an open direction.
2. Sourcing free-space policies from non-robot data. The paper uses motion planning for free-space motion, but suggests inexpensive large-scale non-robot data such as human videos as an alternative, raising the question of how to balance, collect, and integrate heterogeneous data sources for modular policies.
3. Extending to tasks with unclear segment boundaries. The authors note that for tasks such as t-shirt folding, sequential boundaries are harder to define, motivating work on policy switching mechanisms.
4. Visual randomization without new robot data. Because the policies operate locally, the authors argue visual randomization may remain tractable through simulation and sim-and-real co-training, potentially without additional real robot data; they also note that pose estimation can be fine-tuned with images and CAD data alone for exotic, reflective, or partially occluded objects.

Target Audience

Researchers and practitioners working on robot learning, contact-rich manipulation, and data-efficient policy training; engineers implementing assembly or insertion automation in industry; and readers interested in modular architectures that combine classical planning with learned components rather than replacing planning end-to-end.

Authors’ abstract

Learned policies trained end-to-end on large datasets often remain brittle in high-precision tasks and struggle with generalization. We find that these limitations largely stem from a lack of structure and focus in data collection. Our key insight is to leverage dense data collection only for the critical segment of contact-rich tasks and to rely on traditional planning during simple free-space motion. We propose an automated data-collection scheme in combination with offline deep reinforcement learning for the critical segment of the task, eliminating reliance on a teleoperator's skill and on online policy updates. Across four challenging real-world tasks, using only 2 to 2.5 hours of autonomous data collection, we achieve an average success rate of 96%, compared to the strongest baseline at 55%. Notably, performance remains high in out-of-distribution scenarios where end-to-end approaches struggle. Our results pave the way for targeted data collection for contact-rich tasks and for high success rates in precision applications.