Integration of the TIAGo Robot into Isaac Sim with Mecanum Drive Modeling and Learned S-Curve Velocity Profiles

AI summary

Overview

Research area: Robotics simulation and sim-to-real transfer, specifically GPU-accelerated physics simulation (NVIDIA Isaac Sim) of an omnidirectional mobile manipulator and the calibration of its mecanum-wheel drive using learned velocity models.

Technical level: Advanced. The paper assumes familiarity with robot kinematics, mecanum wheel mechanics, physics engines, ROS 2, and neural network training.

Scope: The paper documents the integration of the PAL Robotics TIAGo++ Omni robot into Isaac Sim, including two alternative omnidirectional drive controllers and a neural-network calibration that reproduces the real robot's S-shaped wheel acceleration profiles.

What This Paper Is About

The TIAGo++ Omni is a dual-arm mobile manipulator with a mecanum-wheeled, omnidirectional base, but it had never been made available in Isaac Sim, and its Gazebo model only approximates omnidirectional driving rather than physically simulating it. The authors build a calibrated Isaac Sim model of the robot whose base motion matches the real hardware, addressing the fact that the manufacturer's internal wheel controller is a black box whose behavior cannot be inspected or directly reproduced. The goal is to give researchers a simulation platform that is accurate enough for both high-fidelity validation and efficient learning-based control.

Key Contributions

1. A physically accurate model of the TIAGo++ Omni in Isaac Sim, including high-fidelity simulation of its mecanum wheels for omnidirectional driving, with custom procedurally generated wheels (15 free-spinning rollers angled at 45 degrees) and roller colliders modeled as six spheres each.
2. A lightweight velocity-based control model that approximates the real robot's motion by directly setting the base velocity through the Isaac Sim API, reducing required physics steps per second from 360 to 60 while remaining sufficiently accurate.
3. A neural network-based calibration approach that approximately aligns both control models with the S-shaped velocity profile of the TIAGo++ using minimal, simple real-world trajectory recordings.
4. A public release of the integration at https://github.com/AIS-Bonn/tiago_isaac, with full robot simulation including joint control and sensor (LiDAR, RGB-D camera) integration with ROS 2. The authors state that, to the best of their knowledge, no publicly available Isaac Sim model currently exists for a dual-arm omnidirectional robot.

Main Findings

• Simulated trajectories closely match the real robot on simple motions. Mean relative error (MRE) and standard deviation (STDRE) were measured for 4-second velocity commands. For the full physical model: x-direction 8.24 +/- 1.37 percent, y-direction 4.61 +/- 5.54 percent, x-y (diagonal) direction 5.68 +/- 2.71 percent, and rotation 4.30 +/- 1.62 percent. For the lightweight model: x-direction 7.36 +/- 3.71 percent, y-direction 3.89 +/- 1.49 percent, x-y 5.16 +/- 3.07 percent, and rotation 2.97 +/- 1.52 percent.
• Diagonal motions generalize without being trained on. Diagonal (x-y) commands were used for evaluation only and were not included in the training data, so the reported errors on them test generalization to combined motions.
• The lightweight model is both accurate and cheaper. Directly setting base velocity avoids roller collision computation (wheels can be treated as dummy components or not modeled at all) and reduces the required physics steps per second from 360 to 60, reflected in higher simulation frame rates.
• The real robot's controller does not enforce proportional acceleration. For combined motions such as a skewed trajectory with v_x = 0.3, v_y = 0.15, v_theta = 0.0 (a trajectory along x = 2y), the four wheels do not follow acceleration curves scaled to their final target velocities, so the learned predictions deviate from the real wheel behavior even though the average velocity of the learned model stays close to the expected value.
• The simulated robot is slightly harder to control than the real robot. Corners of a square trajectory (a 90-degree change in movement direction) are navigated much more smoothly in simulation than on the real robot, which the authors interpret as artificial noise that can improve robustness.
• Errors accumulate over longer trajectories. Accumulated error becomes more significant for longer and more complex trajectories, particularly where acceleration patterns such as simultaneous x-y movement are modeled less accurately; the authors argue this is unlikely to be a major issue because real-time closed-loop trajectory controllers continuously correct errors.
• S-curve modeling required a custom function. Standard S-curves such as the logistic function did not capture the observed behavior; the velocity profiles show asymmetrical sharpness with a steeper initial rise, a smoother asymptotic approach to final velocity, and a linear transition in between, motivating a three-segment custom model.

Methodology in Plain English

The researchers imported the TIAGo++ Omni's non-wheel joints (torso, arms, fingers, head) through the Isaac Sim URDF importer and handled joint control through ROS 2 and the Isaac Sim API, recording and publishing sensor data to ROS 2 topics for visualization in RViz 2. The drive was the hard part, so they built two alternatives.

The first is a faithful physics simulation. The wheels in the URDF file were replaced with custom-generated mecanum wheels containing 15 rollers angled at 45 degrees and scaled to the original size, because proper roller alignment and a precisely round wheel shape are needed to avoid loss of ground contact and jumps in driving behavior. Each roller's collision geometry is represented by six spheres, following prior work, which is smoother and far cheaper than mesh-based or low-poly collision models. A holonomic controller converts a desired Twist command (linear velocities v_x and v_y, rotational velocity v_theta) into four wheel angular velocities using the standard matrix relation involving the wheel radius r and the distances L_x and L_y from the robot's center to the wheels.

Because that matrix says nothing about how fast the wheels actually accelerate, and the manufacturer's controller is a black box, the authors recorded real movements and learned the acceleration behavior. They trained a small neural network (a multi-layer perceptron with layer sizes [1, 35, 15, 5] and softplus activations) to output five parameters (a, b, m, k1, k2) of a custom S-shaped curve directly from a wheel velocity change. The curve has three pieces: a smooth softplus-like ramp, a linear middle segment, and a flipped ramp at the end, with continuity constraints enforced during training so the pieces meet at a and b and asymptotically share the slope m. Training used the Adam optimizer, and 100 network instances with different random initializations were trained, with the best selected by lowest validation error.

At runtime, when the robot must go from its current velocity to a new commanded velocity, the model predicts an S-curve for the wheel velocity difference (computed with a conservative bound derived in the appendix using the triangle inequality) and uses it as a fraction p_t to interpolate the Twist command at each timestep. The second, lightweight controller skips the wheel physics entirely, directly setting the base velocity via the Isaac Sim API while still applying the S-shaped profile.

Validation used open-loop trajectories recorded on both the real and simulated robot. Real-world pose was measured with OpenVR using a VIVE tracker mounted at the robot's rotational center, triangulated by two VIVE lighthouses. The evaluation set covered 20 target velocities from 0.05 to 1.00 m/s in the x-, y-, and x-y-directions plus rotational velocities around the z-axis from 0.05 to 1.5 rad/s; each command lasted four seconds, was recorded with the robot's onboard encoders, and was repeated three times and averaged.

Why This Matters

This work addresses a practical gap: a widely used research robot had no Isaac Sim model, and its omnidirectional base could not simply be imported because mecanum wheels require custom physics and the commercial controller's acceleration behavior is not documented. The paper contributes a calibration recipe that uses only a small number of simple motion recordings, and a design insight (proportional acceleration across wheels) that is relevant to anyone building controllers for mecanum bases.

Real-world applications:

• Mobile manipulation tasks such as grasping and assembly, which the authors cite as applications accelerated by efficient physics simulation.
• Logistics and service robotics, which the authors list as industries spanned by mobile manipulator applications.
• Healthcare and human-robot collaboration, also listed among the application areas of general-purpose mobile manipulators.
• Object retrieval in diverse environments, which the authors cite as a representative capability of combined mobility and manipulation.

Industry relevance: GPU-accelerated simulators are positioned as the backbone of learning-based robotics pipelines, and Isaac Lab integration lets researchers apply state-of-the-art reinforcement learning algorithms to such platforms. The lightweight model is described as well suited to high-throughput tasks such as reinforcement learning, while the physically accurate model suits fine-tuning and validation when precise behavior is required. The lightweight approach also generalizes to any holonomic robot, independent of how the real PID controller is implemented, and removes the need to tune wheel joint physics parameters in Isaac Sim.

Future Directions

• Further validating the simulation framework for tasks such as reinforcement learning and trajectory tracking.
• Exploring alternative approaches to improve the modeling of S-shaped velocity profiles for mecanum-wheeled robots.
• Determining whether accurately modeling the acceleration profile of an arbitrary black-box mecanum robot controller is feasible at all, since it would require collecting extensive real-world data for transitions between all possible velocity states and fitting a per-wheel model.
• Having future work that uses this model confirm whether the current level of approximation is sufficient for real use cases.
• The authors also note that enforcing proportional acceleration in the original controller would require reimplementing it, a task that falls to PAL Robotics since the authors lack access to that level of hardware control.

Target Audience

Robotics researchers and engineers who work with Isaac Sim, Isaac Lab, or reinforcement learning on mobile manipulators; developers building simulation models of holonomic or mecanum-wheeled platforms; and practitioners interested in sim-to-real calibration from limited real-world data. It is most useful to readers already comfortable with robot kinematics, ROS 2, and neural network training, and least accessible to beginners looking for an introductory treatment of simulation.

Authors’ abstract

Efficient physics simulation has significantly accelerated research progress in robotics applications such as grasping and assembly. The advent of GPU-accelerated simulation frameworks like Isaac Sim has particularly empowered learning-based methods, enabling them to tackle increasingly complex tasks. The PAL Robotics TIAGo++ Omni is a versatile mobile manipulator equipped with a mecanum-wheeled base, allowing omnidirectional movement and a wide range of task capabilities. However, until now, no model of the robot has been available in Isaac Sim. In this paper, we introduce such a model, calibrated to approximate the behavior of the real robot, with a focus on its omnidirectional drive dynamics. We present two control models for the omnidirectional drive: a physically accurate model that replicates real-world wheel dynamics and a lightweight velocity-based model optimized for learning-based applications. With these models, we introduce a learning-based calibration approach to approximate the real robot's S-shaped velocity profile using minimal trajectory data recordings. This simulation should allow researchers to experiment with the robot and perform efficient learning-based control in diverse environments. We provide the integration publicly at https://github.com/AIS-Bonn/tiago_isaac.