Group Inertial Poser: Multi-Person Pose and Global Translation from Sparse Inertial Sensors and Ultra-Wideband Ranging

AI summary

Overview

Research area: Wearable human motion capture — specifically multi-person 3D pose and global translation estimation using sparse inertial measurement units (IMUs) fused with ultra-wideband (UWB) ranging.

Technical level: Advanced. The paper assumes familiarity with state-space models (S4), SMPL body representations, forward kinematics, and optimization-based inference.

Scope: This paper presents Group Inertial Poser (GIP), an IMU+UWB pipeline that jointly estimates full-body poses and shared-world-frame trajectories for multiple interacting people, along with the first dataset (GIP-DB) for two-person IMU+UWB tracking.

What This Paper Is About

Existing IMU-based motion capture works well for single people, but inertial sensors only measure their own motion — they carry no information about where other people are or how far away they are. This makes it impossible to recover the relative positions and global trajectories of multiple interacting individuals, which is essential for capturing human-to-human dynamics like conversations, handshakes, or dancing. The paper solves this by adding UWB radios to each body-worn sensor, using the measured distances between sensors on different people as an external spatial anchor that constrains and corrects drift in both individual pose and multi-person translation.

Key Contributions

1. GIP, an IMU+UWB method for multi-person motion capture. The first approach to estimate 3D full-body poses and global translations for multiple people in a shared world frame using only sparse wearable sensors plus inter-sensor distances. It achieves state-of-the-art accuracy against prior inertial methods.

2. A structured state-space model (SSM) backbone for pose estimation. The authors adapt S4-style sequence models to inertial human motion estimation for the first time, replacing LSTMs and improving long-range temporal modeling and accuracy.

3. A two-step optimization for translation. An initial position optimizer determines each person's starting position in a shared frame (no calibration or manual alignment needed — users can start anywhere), followed by a trajectory optimizer that refines the full path using between-person distance constraints plus velocity/acceleration smoothness terms.

4. GIP-DB, the first IMU+UWB two-person dataset. Over 200 minutes of recordings from 14 participants (7 pairs) performing everyday and interactive activities, with synchronized IMU signals, UWB distances, OptiTrack ground-truth translation, and SMPL pose parameters from an Xsens suit.

Main Findings

• Large gains on synthetic two-person data (InterHuman): GIP reduces full-body joint angle error by 22% versus UIP and 33% versus PIP, and cuts relative-translation distance error at 20 seconds to ~2 cm, versus ~60–80 cm for baselines.

• Robust under real-world noise (GIP-DB): Despite real UWB measurements being far noisier than simulated ones (average RMSE of 5 cm for same-person, 15 cm for between-person ranging due to body occlusion), GIP reduces the 20-second distance error by 72% relative to the baselines. Fine-tuning on GIP-DB further reduces angle error from 23.3° to 17.6°.

• Initial position optimization is critical: Removing it causes translation error at 6 m to jump from ~99 cm to ~116 cm in the ablation, showing that optimizing the full trajectory without a stable initialization converges poorly. Using the optimizer matches ground-truth initialization almost exactly.

• SSMs beat LSTMs: Swapping the state-space modules for LSTMs degrades both angle error (9.94° → 12.88°) and translation error, validating the architectural choice.

• Accuracy improves with more people: In a synthetic four-person test, translation error drops from 2.34 m (1 person) to 1.52 m (2), 1.20 m (3), and 1.15 m (4), because additional people provide more spatial constraints.

Methodology in Plain English

Each person wears six small sensors — on the head, pelvis, both wrists, and both knees — and each sensor contains both an IMU (measuring orientation and acceleration) and a UWB radio (measuring distances to every other sensor).

The pipeline has three stages. First, individual pose estimation: a neural network built on state-space sequence modules processes each person's accelerations, orientations, and within-person sensor distances to predict SMPL body pose parameters and a trajectory relative to that person's own starting point. A graph convolutional network handles the distance information, and a physics-based optimizer enforces plausible joint angles and foot contacts.

Second, initial position optimization: since each person's trajectory is in their own private coordinate frame, the system must figure out where person 2 starts relative to person 1. It does this by computing what the between-person sensor distances would be given the estimated poses at each frame, and searching for the starting offset that best matches the actual UWB measurements over the whole sequence.

Third, trajectory optimization: with everyone now in a common frame, the system refines all trajectories simultaneously, again minimizing the mismatch between predicted and measured between-person distances, while penalizing jerky velocity and acceleration changes so the motion stays smooth. Both optimization steps use L-BFGS.

Why This Matters

Impact on research: This work closes a gap that has kept inertial motion capture confined to single-person scenarios. By showing that UWB distances between body-worn sensors are sufficient to anchor multiple people in a shared frame, it establishes IMU+UWB as a viable modality for studying social interaction — a domain previously dominated by multi-camera systems that fail under occlusion or in the wild. The SSM-based pose estimator also demonstrates that modern sequence architectures transfer well to inertial motion, which may influence future mocap model design.

Real-world applications:

• Social and behavioral science: capturing natural, unscripted interactions outdoors or in homes without camera coverage.
• VR/AR multi-user experiences: tracking several participants in the same virtual space without external base stations or marker setups.
• Sports and rehabilitation: monitoring athletes or patients in group training, where relative positioning and coordination matter.
• Robotics and human-robot collaboration: equipping workers or robot operators with sensors that report both pose and position relative to others.

Industry relevance: The sensors used (DWM3000 UWB, LSM6DSL IMU) are commercially available and increasingly present in smartphones, watches, and tags. This points toward consumer-grade full-body multi-person tracking that requires no infrastructure — attractive to gaming, telepresence, healthcare, and industrial safety markets where camera-based systems are impractical or privacy-invasive.

Future Directions

• Mitigating UWB noise under occlusion. Non-line-of-sight conditions between people produce the dominant error source; better filtering or channel impulse response analysis could substantially improve accuracy.
• Per-person body shape estimation. The current model assumes an average body shape, ignoring variation between individuals, which affects distance predictions and pose plausibility.
• Reducing computational cost. Optimization-based inference takes ~2 seconds for a 30-second sequence — fast enough offline, but a barrier for live, on-device deployment.
• Eliminating foot sliding. The trajectory optimization can introduce foot slip that is not currently corrected, a common quality issue in inertial mocap.

Target Audience

Researchers and graduate students in computer vision, wearable computing, and ubiquitous sensing; engineers building VR/AR, motion capture, or human-robot interaction systems; and practitioners in biomechanics or behavioral science who need markerless, infrastructure-free multi-person tracking. Readers should be comfortable with deep learning architectures, optimization, and the SMPL body model — the paper is not beginner-friendly, though the high-level pipeline is accessible.

Authors’ abstract

Tracking human full-body motion using sparse wearable inertial measurement units (IMUs) overcomes the limitations of occlusion and instrumentation of the environment inherent in vision-based approaches. However, purely IMU-based tracking compromises translation estimates and accurate relative positioning between individuals, as inertial cues are inherently self-referential and provide no direct spatial reference for others. In this paper, we present a novel approach for robustly estimating body poses and global translation for multiple individuals by leveraging the distances between sparse wearable sensors - both on each individual and across multiple individuals. Our method Group Inertial Poser estimates these absolute distances between pairs of sensors from ultra-wideband ranging (UWB) and fuses them with inertial observations as input into structured state-space models to integrate temporal motion patterns for precise 3D pose estimation. Our novel two-step optimization further leverages the estimated distances for accurately tracking people's global trajectories through the world. We also introduce GIP-DB, the first IMU+UWB dataset for two-person tracking, which comprises 200 minutes of motion recordings from 14 participants. In our evaluation, Group Inertial Poser outperforms previous state-of-the-art methods in accuracy and robustness across synthetic and real-world data, showing the promise of IMU+UWB-based multi-human motion capture in the wild. Code, models, dataset: https://github.com/eth-siplab/GroupInertialPoser