IMUDINO: Self-distilled Foundation Model for Inertial Sensing

AI summary

Overview

Research area: Machine learning — self-supervised representation learning for inertial measurement unit (IMU) / wearable motion sensor data, with applications spanning human activity recognition, health and gait analysis, motion capture, and robot odometry.

Technical level: Intermediate. The paper assumes familiarity with self-supervised learning concepts (student–teacher distillation, masked token objectives, EMA teachers) and transformer architectures, but its central argument is accessible without deep mathematics.

Scope (one sentence): The paper proposes IMUDINO, a DINO-style self-distilled foundation model for inertial sensing that produces both global motion descriptors and dense patch-level features from a minimalist transformer, and evaluates it across four task families including out-of-distribution robot odometry.

What This Paper Is About

Inertial sensing — data from accelerometers and gyroscopes in phones, wearables, and robots — underpins many tasks, from recognizing activities to tracking a robot's trajectory. Existing foundation models for this modality are fragmented: most target only human activity recognition (HAR), many require fixed input length and sampling rate, they generalize poorly to unseen sensor locations, and some are outperformed by plain supervised models. The authors aim to build a single general-purpose inertial foundation model that stays competitive across both global tasks (HAR, health) and dense tasks (motion capture, odometry), while natively handling varying input lengths, sampling rates, and sensor placements.

Key Contributions

1. Adaptation of DINO self-distillation to inertial data. Rather than contrastive learning, masked reconstruction, or augmentation prediction — the three lineages the paper traces through prior inertial self-supervised work — the authors build a student–teacher setup combining a DINO global objective, an iBOT++ patch-level objective, and a KoLeo regularizer, all driven by a set of "semantics-preserving motion transformations" designed specifically for IMU signals.

2. A minimal 1D transformer (IMUFormer) with frequency-aware positional encoding. Starting from a standard ViT, the authors make only two changes: a 1D non-overlapping convolution patchify layer, and a sinusoidal positional encoding that rescales positions by p̃_b = p · f_ref / f_b so that equal elapsed time gets equal encoding regardless of sampling rate (with β = 100 and f_ref = 100 Hz). This parameter-free change provides native support for variable input length and sampling rate.

3. A single model family bridging global and dense tasks. The paper reports the first inertial foundation model claimed to be competitive on HAR, health, motion capture, and robot odometry simultaneously, whereas prior work targeted one subfield — and it claims to be the first to show frozen-backbone cross-embodiment transfer to robot inertial odometry.

4. Data efficiency. Training uses a curated 11.6k-hour corpus (approximately 500 days) drawn from Capture24, SHL, and PAAWS-Lab, which the authors contrast with the roughly 700,000 days reported for prior work by Yuan et al. (2024) and Xu et al. (2025), and with the "700k–800k person-days" they attribute to RelCon and Yuan et al.

Main Findings

• HAR: state-of-the-art with a 2-layer MLP probe. Under leave-one-subject-out cross-validation and macro F1, IMUDINO-85M averages 89.73 across seven HAR datasets (MotionSense 98.54, HAPT 89.02, HHAR 96.00, DSADS 94.13, WEAR 92.34, OPPORTUNITY 58.40, Shoaib 99.68). This exceeds the best supervised baseline TSF (86.76), DeepConvLSTM (80.57), Attend & Discriminate (82.74), and IF-ConvTransformer (85.59). It also exceeds the self-supervised baselines SimCLR (83.86), Yuan et al. (74.36), RelCon (69.64), and HAR-FM (73.20), and the general time-series foundation models TimesFM 3.0 (73.29) and Chronos-2 (66.36). The 6M variant already averages 88.35.

• Motion capture: competitive without physical priors. On AnDy, UNIPD, CIP, and Virginia Natural Motion, a frozen MLP probe gives macro-average SIP 14.65°, angular error 5.34°, position error 4.71 cm — described as roughly TIP-level (13.77° / 5.55° / 4.64). A finetuned backbone with an MLP head reaches 12.28° / 4.63° / 3.95 cm, better than PIP (12.67° / 5.17° / 4.25) but still behind DynaIP (10.86° / 4.49° / 3.79) — a gap the authors attribute to the absence of physical priors and to using synthesized rather than raw gyroscope channels.

• Robot odometry: out-of-distribution transfer beats the specialist. With a frozen backbone evaluated platform-agnostically on the 2026 TartanIMU Challenge validation split, IMUDINO-85M + Transformer achieves AVE 0.209, ATE20 0.854, and score 0.280, versus TartanIMU's platform-agnostic 0.721 / 2.206 / 0.872 and its platform-expert 0.350 / 1.116 / 0.429. The authors emphasize that IMUDINO never saw robot IMU data during training.

• Ablation: self-supervised pretraining carries most of the gain. A randomly initialized frozen backbone collapses to 65.32 mF1 on HAR (vs. 89.73 for IMUDINO-85M), and a randomly initialized trainable backbone reaches 83.70 — still below the pretrained model. Removing the patch-level objective drops HAR to 89.41 and worsens MoCap and odometry; removing rotation augmentation drops HAR to 88.66; removing frequency-aware positional encoding drops HAR to 89.60 and odometry to 0.260/1.022/0.343. Removing KoLeo slightly improves MoCap but lowers HAR to 89.40 and reduces training stability.

• Global and dense features are deliberately decoupled. A token-similarity analysis (aggregated over 700 examples from 7 HAR datasets) shows that rotation and resampling augmentations make CLS tokens nearly invariant to those perturbations, while patch tokens remain sensitive — dropping to near-random similarity at 180° rotations — because invariance-inducing transformations are applied only to the global objective. This is how the paper resolves the tension between semantic invariance and local detail.

• Feature structure is interpretable. t-SNE visualizations show separable activity clusters and, within the MotionSense walking class, distinct per-user clusters — evidence that IMUDINO captures individual gait differences, which the authors connect to a gait recognition experiment in the appendix.

• Health and gait analysis. The paper states that IMUDINO features are competitive on four health and gait tasks (activity intensity estimation, fall detection, gait recognition) plus an online open-set HAR experiment; the numeric results for these appear in appendices that are not included in the provided text, so they are not reported here.

Methodology in Plain English

The recipe rests on one idea from computer vision: if you show a "teacher" network and a "student" network two different distorted views of the same input, and train the student to match the teacher's output distribution, the network learns representations that ignore the distortions. The teacher is not trained by gradients; it is a slowly updated exponential moving average of the student.

For IMU data, the distortions must change nuisance factors while leaving the meaning of the motion intact. The authors pick six:

• Cropping — global views covering 50–80% of a sample and local views covering 5–40%, on the logic that actions close in time are semantically related.
• Scaling — magnitude factors and temporal (speed) factors both drawn from 0.8–1.2, applied with probability 0.5, because the same action is performed at different intensities and paces.
• Rotation — random SO(3) rotations with probability 0.5, so the representation captures the motion rather than the sensor's mounting orientation.
• Resampling — new frequency drawn from 30–200 Hz with probability 0.5, since 50 Hz and 200 Hz recordings of the same motion mean the same thing.
• Channel masking — each channel zeroed with probability 0.2, so the model does not over-rely on gyroscope data.

The training loss has three terms with weights λ_d = λ_i = 1.0 and λ_k = 0.2: a DINO global loss on class tokens (with sharpening and Sinkhorn-Knopp centering on the teacher), an iBOT++ patch-level loss where the student sees masked tokens but the loss is applied to all tokens (masked and visible), and a KoLeo regularizer that spreads normalized class-token features apart within a batch.

Architecturally the model is deliberately plain: patch size 16, four register tokens, GELU MLPs with ratio 4, offered in 2.7M, 6M, 25M, and 85M parameter sizes (main experiments use 6M and 85M, the latter being ViT-B with 24 attention heads instead of 12). Training uses non-overlapping 1024-frame windows, 2 global and 6 local views per sample, batch size 1536 (yielding 3072 global and 9216 local views), 60 epochs on 6 Nvidia H100 GPUs, AdamW with learning rate 5e-4 and weight decay 5e-2 — 11 hours for the 85M model and 6 hours for the 6M model.

Data curation follows four steps (file-level quality control, unit normalization, cropping into 1024-frame samples, sample-level quality control). Capture24 contributes 2410 hours of triaxial acceleration after filtering from 3883 original hours (151 participants, 100 Hz, gyroscope channels zero-filled); SHL contributes 1138 hours of 6-axis data from four phone locations; PAAWS-Lab contributes 8069 hours from 252 participants and 20 IMUs after removing static segments.

Why This Matters

The paper argues that inertial foundation models have lagged vision and language models not because the modality is unsuitable, but because the field split into narrow sub-communities — HAR models that sometimes lose to supervised baselines, and odometry models that need dense features — with little crossover. IMUDINO is a concrete demonstration that one frozen backbone can serve both, and it does so with a simpler architecture and far less pretraining data than prior work claims.

Real-world applications:

• Wearable health monitoring — activity intensity estimation, fall detection, and gait recognition using frozen features, relevant to elder care and clinical assessment.
• Consumer motion capture — full-body pose estimation from body-worn IMUs without camera infrastructure, for sports, rehabilitation, or animation, using a simple MLP or transformer decoder.
• Robot and vehicle navigation — inertial odometry for cars, quadrupeds, drones, and humans, where the model transfers to embodiments it never saw in training.
• Smartphone and smartwatch sensing — because the model tolerates variable sampling rates, variable window lengths, and sensor placements not seen during pretraining, it suits heterogeneous consumer hardware.

Industry relevance: Vendors of wearables, smartphones, robotics platforms, and AR/VR systems currently train task-specific models per device and per sensor placement. A frozen, general-purpose backbone with a lightweight probe dramatically lowers the labeling and engineering cost per new product task, and the reported 6-hour training time for the 6M model makes the recipe reproducible at modest compute.

Future Directions

• Closing the motion-capture gap. IMUDINO trails DynaIP on pose estimation. The authors suggest that pairing its dense features with "carefully crafted, physics-informed decoder architectures and training objectives" is the natural next step, and their own comparison is confounded by having to synthesize gyroscope channels.
• Scaling the pretraining corpus and model. The paper explicitly frames 11.6k hours as "relatively modest" relative to the hundreds of thousands of person-days used elsewhere, and includes a scaling experiment (Appendix H, not in the provided text). How far performance rises with more data and larger IMUFormer variants remains open.
• Resolving the global/dense trade-off more cleanly. KoLeo and channel masking improved HAR and stability but hurt MoCap metrics, and frequency-aware positional encoding helped HAR and odometry while slightly hindering MoCap. Whether these tensions can be eliminated rather than traded off is unresolved.
• Extending beyond the tested domains. The open-set online HAR experiment and the health/gait results are reported only in appendices here; broader validation on clinical, industrial, and multi-modal (IMU plus video or EMG) settings is a logical follow-on. The full test split of the TartanIMU Challenge was also unreleased at the time of writing, so the odometry result rests on the official validation split.

Target Audience

Researchers and engineers working on wearable sensing, mobile health, human activity recognition, and robot state estimation — particularly those who want a strong frozen backbone to attach lightweight task heads to. It is also valuable for self-supervised learning researchers interested in how a vision-native recipe (DINO/DINOv2/iBOT++) translates to a non-visual, physically grounded time-series modality, since the paper's most transferable insight is that augmentation design, not architecture, determines what invariances a foundation model learns. Readers need only moderate background in deep learning; the paper is less suited to those seeking a production-ready, fully benchmarked system, since several supporting experiments live in appendices and one benchmark split is not yet publicly scored.

Authors’ abstract

We present IMUDINO, the first foundation model (FM) for inertial sensing to achieve competitive performance across diverse sensing tasks including human activity recognition (HAR), health, motion capture, and odometry. Prior inertial FMs often focus on one subfield of inertial sensing and sometimes underperform state-of-the-art supervised methods (e.g., in HAR), making their scope considerably narrower than FMs for vision and natural language. Moreover, they often require fixed input length and sampling rate, and generalize poorly to unseen sensor locations. To build a general-purpose foundation model for inertial sensing, we adapt the DINO self-supervised learning (SSL) paradigm to inertial data through a set of semantics-preserving motion transformations that simulate variations in motion pattern and sensor setup. For model architecture, we use a plain transformer encoder with frequency-aware positional encoding that natively supports varying input length and sampling rate. This simple setup proved remarkably effective for learning global and dense features generalizable across users, sensor setups, tasks, and, in the case of robot odometry, embodiments. We hope IMUDINO can serve as a backbone for inertial sensing and kickstart the development of general-purpose inertial FMs.