Manipulation of Deformable Linear Objects Using Model Predictive Path Integral Control with Bidirectional Long Short-Term Memory Learning

AI summary

Overview

Research area: Robotics — specifically robotic manipulation of deformable linear objects (DLOs) such as cables, ropes, and wires, combining learned dynamics models with sampling-based optimal control.

Technical level: Intermediate. The paper assumes familiarity with Model Predictive Control, recurrent neural networks (LSTM), and basic cost-function notation, but explains the concepts enough for a reader with a general machine-learning or control background.

Scope: The paper presents and evaluates a framework that pairs a bidirectional LSTM dynamics model trained on MuJoCo simulation data with a Model Predictive Path Integral (MPPI) controller to shape cables and wires into target configurations, validated both in simulation and on a Franka Emika Panda robot.

What This Paper Is About

Deformable linear objects have effectively infinite degrees of freedom, so a robot force applied to them changes both their position and their shape in nonlinear ways that are hard to model in real time. The authors' goal is to build a data-driven dynamics model of the DLO (a bidirectional LSTM trained on synthetic MuJoCo data) and use it inside an MPPI controller to plan and execute robot motions that drive the object into a desired target shape. They test the approach in simulation and in real robot experiments across three scenarios.

Key Contributions

1. The authors contribute datasets, a model architecture, and model weights for cable modeling, publicly released at https://doi.org/10.18419/DARUS-5050.
2. They propose a framework for DLO manipulation that uses a Model Predictive Path Integral controller together with a learned biLSTM dynamics model, including the 2D relative-position state representation that makes the model translation-invariant.
3. They demonstrate the effectiveness of the method in simulation and in real-world experiments, including a hyperparameter study (random search followed by grid search) to identify the best biLSTM configuration.
4. They extend prior work by applying MPPI control to manipulate different types of DLOs.

Main Findings

• biLSTM prediction accuracy: Evaluated over 100 rollouts of 150 time steps (3 seconds), the model achieved an average shape error of 3.3 cm and an average velocity error of 61.59%. The paper notes these are similar to results in prior work and that error grows with the number of time steps, though shape structure, length, and curvature were preserved in qualitative results at 1, 50, and 150 time steps.

• Best hyperparameters: The best-performing model used a learning rate of 1e-4, weight decay of 1e-7, batch size 128, and three biLSTM layers with 512 neurons each (equivalent to a hidden layer size of 256 neurons in each direction). The random search found top configurations commonly used hidden layer sizes of 256 or 512, up to 100 epochs, learning rates between 1e-5 and 1e-3, weight decay between 1e-7 and 1e-5, and 2–6 biLSTM layers; training beyond 50 epochs did not yield significant improvements.

• Simulation, U-shape target: Across 100 trials with a success criterion of maximum 2 cm deviation between simulated capsule positions and target points, the success rate was 93%, with an average time of 7.26 s per successful trial (success defined as reaching the U-shape within 30 seconds).

• Simulation, random goal shapes: Across 1,000 trials of shaping the DLO into random goal shapes, the success rate dropped to 20.5%, with an average time of 13.3 s per successful trial.

• Experiment scenario 1 (2D, 6 mm cable, 9 markers): Across 20 trials, the success rate was 85%, with an average of 15.7 s per successful trial. The fastest trial took 2.1 s and the slowest took 45.2 s.

• Experiment scenario 2 (3D, same cable on a 7 cm platform): Across 20 trials, the success rate was 50%, with an average of 28.3 s per successful trial. The fastest trial took 10.4 s and the slowest 80.5 s. The authors note this scenario was not trained for, the control process was less robust, and times were significantly higher.

• Experiment scenario 3 (2D, 1.5 mm wire, no markers, FastDLO shape estimation): This scenario was included to test generalization of the biLSTM model, and the paper states it was less robust than the first scenario. The quantitative results are not reported in the provided paper content — the text ends mid-sentence.

• Robustness observations: The approach transfers to new scenarios because training used relative capsule positions with respect to the end effector. However, re-optimizing MPPI parameters is necessary when the target shape differs significantly from the one the controller was tuned for, and additional parameterization is required for severe DLO deformations.

Methodology in Plain English

The authors split their system into two parts: a model of how the cable moves, and a controller that decides how the robot should move.

Building the model. They simulated a 0.5 m cable in MuJoCo as a chain of 50 capsule-shaped mass points connected by linear, torsional, and bending springs, with a Young's modulus of 4×10^6 Pa, a shear modulus of 1×10^6 Pa, and damping of 1 Nms/rad. Gravity, friction, and air drag were not simulated. The cable was clamped at one end; the other end was moved by a simulated robot arm along random trajectories at a height of 0.15 m, with target positions sampled from x ∈ [0.05, 0.35] m and y ∈ [-0.2, 0.2] m, and random rotations about the z-axis in ψ ∈ [-1, 1] rad. The range was chosen to avoid overstretching the cable. Each trajectory lasted 5 s with a 1 ms simulation step. To keep computation manageable, only every fifth capsule was kept (10 capsules after removing the first and last), and the state was expressed as relative positions with respect to the end effector so the network learns deformation rather than absolute location. The state also includes the end effector's position and velocity (x, y, z, ψ and their rates).

Training. The biLSTM was trained on 10,000 trajectories split 80% training / 20% testing, using the Adam optimizer and mean squared error loss on node velocities. The authors first ran a random search to narrow the hyperparameter space, then a grid search on a smaller range, selecting the configuration with the best validation loss.

Controlling. The controller is MPPI, a sampling-based variant of Model Predictive Control. At each step it samples many candidate control sequences by perturbing a nominal sequence with pink noise, rolls out each candidate through the biLSTM model, and scores it with a cost function combining a shape cost (weighted deviation from target capsule positions, weighted by matrix Q) and a control cost (weighted input effort, weighted by matrix R). Trajectories with lower cost receive exponentially higher weight according to a temperature parameter λ, and the nominal control sequence is updated as the weighted average of the disturbances. Only the first control input is applied to the robot; the rest is discarded, and the process repeats with warm-starting from the previous solution.

Testing. Simulation used an MPPI horizon of 5, time increment 0.2, 20 samples, λ = 0.002, disturbance standard deviation [0.5; 0.5; 0; 3] for [X, Y, Z, Rot(Z)], Q = 150, R = 5. Experiments used a horizon of 8, time increment 0.02, 200 samples, λ = 0.002, disturbances [0.2; 0.2; 0; 0.02], Q = 150, R = 5. The real setup used a Franka Emika Panda robot, an Intel RealSense D435i RGB-D camera (40 fps), a communication frequency of 1,000 Hz, and an Ubuntu 24.04 real-time desktop computer. Cable shape in the marked scenarios was tracked with a color filter on 9 markers; the unmarked wire scenario used the FastDLO algorithm for shape estimation with 9 virtual markers placed along the tracked shape. All three experiment scenarios aimed for a U-shape with a maximum 2 cm deviation from target.

Why This Matters

Impact on research. The work shows a concrete path for combining a learned sequence model with sampling-based control for objects whose dynamics resist first-principles modeling in real time. Releasing the dataset, architecture, and weights gives other researchers a reproducible baseline. The translation-invariant relative-position representation is a practical trick that lets a model trained in one task setup transfer to another.

Real-world applications:

• Wire harness routing and connector insertion in electronics and automotive assembly.
• Cable handling and laying in industrial installation, where robots must place flexible cables into specific shapes.
• Hose or rope manipulation in logistics, agriculture, and construction.
• Deformable object handling in service and household robotics, such as untangling or arranging cables and cords.

Industry relevance. Robotic assembly and production lines increasingly mix rigid and flexible parts; automated handling of flexible objects by robots remains a research problem the paper describes as not yet entirely solved. The reported 85% success rate on a 2D marked cable task and the 93% simulation success rate indicate the approach is credible for structured tasks, while the 50% rate in the untrained 3D scenario and the 20.5% rate on random goal shapes show where further engineering work is needed. The paper also notes parameterization and computational demand as practical constraints for deployment.

Future Directions

1. Improve MPPI sampling efficiency. The authors suggest that an adaptive sampling approach, where exploration during construction of the search tree focuses only on trajectories with high solution potential, might improve results.
2. Increase compute for larger sample counts. The paper identifies the computational demand of the biLSTM model as a limiting factor, especially when processing many samples, and states that increasing computing resources could improve accuracy and robustness since larger sample sizes generally help.
3. Generalize across target shapes and materials. Re-optimizing MPPI parameters was necessary when target shapes differed significantly from the tuning shape, and severe deformations required additional parameterization. Extending the method to handle these without retuning is an open problem.
4. Extend evaluation to untrained and markerless real-world scenarios. The 3D scenario and the 1.5 mm wire scenario (the latter's quantitative results are not reported in the provided content) point to a need for more systematic testing of generalization, including broader DLO types and material properties beyond the cables and wire used here.

Target Audience

This paper is most useful to robotics researchers and graduate students working on deformable object manipulation, learning-based control, and model predictive control. It is also relevant to industrial automation engineers evaluating practical approaches to flexible-part handling, and to practitioners interested in MPPI as an alternative to gradient-based MPC for systems with learned, non-differentiable or uncertain dynamics. Readers without a background in control theory or recurrent networks will find the framework conceptually clear, but the parameter tables, cost functions, and evaluation metrics assume intermediate technical familiarity.

Authors’ abstract

The manipulation of Deformable Linear Objects (DLOs) such as cables poses a significant challenge for automation due to their infinite degrees of freedom and non-linear dynamics. In this paper we present a machine learning based optimal control approach for the manipulation of DLOs. This approach is divided into two main components: modeling and control. For modeling the dynamics of the DLO, we propose a learning based approach using a bidirectional Long Short-Term Memory (biLSTM) network. The biLSTM network is trained on synthetic data generated by the MuJoCo physics engine. For manipulating the DLO, a model predictive control strategy that employs Model Predictive Path Integral (MPPI) control is selected. The proposed approach is evaluated through simulation and experiments. The results demonstrate the effectiveness of the proposed method in achieving accurate and efficient manipulation of DLOs.