PressTrack-HMR: Pressure-Based Top-Down Multi-Person Global Human Mesh Recovery

AI summary

Overview

Research area: Computer vision and ubiquitous sensing — specifically, contact-based (pressure mat) human pose estimation and global human mesh recovery (HMR) extended from single-person to multi-person scenarios.

Technical level: Advanced. The paper assumes familiarity with multi-object tracking (MOT), the SMPL body model, transformers, and 3D human pose estimation metrics.

Scope: The paper presents a two-stage top-down pipeline that recovers per-person global human meshes and world trajectories from floor pressure signals alone, together with a new multi-person interaction pressure dataset (MIP) and a footprint-tracking method (PressTrack) tailored to pressure data.

What This Paper Is About

Existing pressure-mat pose estimation works only handle one person at a time; when several people walk randomly on the same mat, their pressure footprints blend into a single jumbled signal. This paper solves the resulting two sub-problems — separating each person's footprint within a frame, and linking those footprints across frames — so that a pose estimator can then reconstruct each individual's full 3D body mesh and global trajectory. The authors also build the first pressure dataset containing genuine multi-person interaction.

Key Contributions

1. PressTrack, a tracking-by-detection method designed specifically for pressure footprints. It introduces a new Union over Enclosure (UoE) similarity metric that replaces IoU and motion-prediction (Kalman filter) cues, which fail on pressure data because detection boxes change size abruptly and move discontinuously between frames.
2. PressTrack-HMR, described as the first pipeline to recover multi-person global human meshes and trajectories exclusively from pressure signals, using a top-down detect → track → per-person HMR structure.
3. The MIP (Multi-person Interaction Pressure) dataset, with over 138K synchronized tactile and visual frames captured from 20 volunteers: 20 single-person sets, 30 randomly composed 2- or 3-person sets, and 2 couple-dance sets, recorded on a 120 × 240 sensing-unit mat (1 cm pitch) with seven synchronized RGB cameras for ground-truth annotation.
4. Automatic footprint label generation using projected foot joints (toe-base and ankle) from RGB-derived ground truth, assigning discrete pressure regions to individuals by nearest-joint distance, then merging into per-person bounding boxes for detector training.

Main Findings

• Tracking performance: PressTrack reaches 93.6% MOTA and 94.8% MOTP, versus 66.1%/79.1% for ByteTrack and 82.5%/87.7% for BoT-SORT. Missed detections, false positives, fragmentations, and ID switches (437 vs. 14,453 for ByteTrack) all drop sharply.
• End-to-end HMR accuracy: With tracked detections, the pipeline achieves 89.2 mm MPJPE and 112.6 mm WA-MPJPE₁₀₀ on unseen sequences — about one ID switch per 523 frames, acceptable because HMR uses 16-frame sliding windows.
• Cascading error from tracking: Using ground-truth boxes instead of tracked boxes improves unseen-sequence MPJPE from 89.2 mm to 81.8 mm, isolating the contribution of tracking error to the final result.
• Generalization to new people: In the unseen-subject split, end-to-end MPJPE rises only 7.6 mm (to 96.8 mm) and WA-MPJPE₁₀₀ only 2.4 mm (to 115.0 mm) relative to unseen sequences, indicating strong generalization to previously unseen individuals.
• Optimal temporal window: MPJPE for the isolated HMR module falls from 94.2 mm at sequence length 1 to 81.8 mm at length 16 (a 12.37 mm reduction), then plateaus or worsens at length 32. The end-to-end pipeline also minimizes at length 16, but with a smaller 6.99 mm gain since tracking errors accumulate with longer windows.
• Pressure-specific motion physics: Pressure footprints show abrupt box-size changes (single-foot vs. double-foot contact) and jump-like discontinuous motion, which is why standard visual trackers underperform and motion-prediction approaches are ineffective.

Methodology in Plain English

The system works in two stages.

Stage 1 — PressTrack (finding and following each person's footprints). A fine-tuned YOLOv11 detector finds bounding boxes around each person's pressure footprints in every frame. Training labels are generated automatically: discrete pressure blobs are found with thresholding, then each blob is assigned to a person based on which person's projected foot joints are nearest. Because a single foot can create two disconnected pressure regions (forefoot and heel, due to the foot arch and shoe sole), all regions belonging to one person are merged into one box.

To link boxes across frames, the authors replace the usual IoU metric with UoE: the area of the union of two boxes divided by the area of the smallest box enclosing both. This is based on the observation that different people's footprints never occupy the same space simultaneously. Detection confidence scores refine the cost matrix, and the Hungarian algorithm finds the optimal assignment. Unmatched low-confidence detections are discarded; unmatched tracks are held as "lost" for several frames to survive brief foot-lift or two-footed-jump gaps, then removed if they never reappear. Each tracked person's pressure patch is cropped, padded to 128 × 128, and paired with the box center coordinates as localization info.

Stage 2 — Human Mesh Recovery. For each person's pressure patch sequence, a ResNet image encoder extracts spatial features, which are concatenated with box-center coordinates and the mat corner coordinates (providing spatial priors). A two-layer Transformer encoder with positional embeddings captures temporal dependencies. Crucially, the model uses an N-to-1 mapping: it averages temporal features and regresses a single mid-frame SMPL mesh (pose θ, shape β, global translation T) via an iterative SMPL regressor. Training minimizes a combination of SMPL parameter error (L1) and 3D joint position error (L2).

Why This Matters

Impact on research: The paper opens a new problem setting — multi-person pressure-based global HMR — and shows it is tractable. It supplies both a needed dataset (MIP, the first with authentic multi-person interaction) and a tracking method designed around the physical properties of pressure footprints, demonstrating that computer-vision tracking baselines transfer poorly to tactile data.

Real-world applications:

• Assisted living and eldercare: Privacy-preserving monitoring of residents' gait, posture, and falls, including shared rooms where multiple people are present.
• Sports and rehabilitation: Continuous multi-athlete movement analysis and injury-risk assessment in clinics or training rooms where cameras are unwanted or impractical.
• Public safety and crowd management: Anonymous crowd dynamics, flow, and congestion analysis in buildings, transport hubs, and event venues.
• Smart environments and VR: Multi-user motion capture for virtual reality and interactive floors without wearing markers or devices.

Industry relevance: Tactile mats are cheap, robust to darkness, immune to occlusion, and inherently privacy-preserving — an attractive alternative to camera arrays for any deployment where lighting, line-of-sight, or privacy regulations constrain RGB sensing. The work is directly relevant to smart-home, healthcare, security, and XR product teams.

Future Directions

1. Diagnosing cascading tracking errors: The authors explicitly plan to analyze how different error types (ID switches versus localization jitter) propagate into HMR accuracy, aiming to reduce tracking-to-pose error accumulation.
2. Closing the ground-truth-versus-tracked gap: Improving footprint localization precision from detection could shrink the ~7 mm MPJPE disparity observed between ground-truth and tracked boxes.
3. Scaling to higher occupancy and denser crowds: The dataset covers only 1–3 people plus couple dance; whether the UoE assumption (non-overlapping footprints) and the tracker's track-management rules hold at much higher pedestrian density remains untested.
4. Improving global trajectory fidelity: RTE is low (0.13–0.96%), but Accel and Jitter roughly double when using tracked detections versus ground truth, so smoothing and motion-smoothness refinement are open problems.

Target Audience

Researchers and graduate students in computer vision, ubiquitous computing, and human motion capture; engineers building tactile-sensing products for healthcare, smart homes, or XR; and dataset/benchmark developers interested in privacy-preserving alternatives to camera-based pose estimation. Readers without a background in multi-object tracking or SMPL-based mesh recovery will find the technical sections demanding.

Authors’ abstract

Multi-person global human mesh recovery (HMR) is crucial for understanding crowd dynamics and interactions. Traditional vision-based HMR methods sometimes face limitations in real-world scenarios due to mutual occlusions, insufficient lighting, and privacy concerns. Human-floor tactile interactions offer an occlusion-free and privacy-friendly alternative for capturing human motion. Existing research indicates that pressure signals acquired from tactile mats can effectively estimate human pose in single-person scenarios. However, when multiple individuals walk randomly on the mat simultaneously, how to distinguish intermingled pressure signals generated by different persons and subsequently acquire individual temporal pressure data remains a pending challenge for extending pressure-based HMR to the multi-person situation. In this paper, we present \textbf{PressTrack-HMR}, a top-down pipeline that recovers multi-person global human meshes solely from pressure signals. This pipeline leverages a tracking-by-detection strategy to first identify and segment each individual's pressure signal from the raw pressure data, and subsequently performs HMR for each extracted individual signal. Furthermore, we build a multi-person interaction pressure dataset \textbf{MIP}, which facilitates further research into pressure-based human motion analysis in multi-person scenarios. Experimental results demonstrate that our method excels in multi-person HMR using pressure data, with 89.2 $mm$ MPJPE and 112.6 $mm$ WA-MPJPE$_{100}$, and these showcase the potential of tactile mats for ubiquitous, privacy-preserving multi-person action recognition. Our dataset & code are available at https://github.com/Jiayue-Yuan/PressTrack-HMR.