REXO: Indoor Multi-View Radar Object Detection via 3D Bounding Box Diffusion

AI summary

Overview

Research area: Indoor radar perception and diffusion-based object detection, specifically multi-view radar heatmap sensing for 3D human detection.

Technical level: Intermediate. Readers need familiarity with object detection basics (bounding boxes, IoU, average precision) and a working idea of diffusion models; background in radar signal processing helps but is not required.

Scope: The paper introduces REXO, a method that performs 3D bounding-box diffusion directly in radar coordinate space to explicitly associate features across horizontal and vertical radar views, reporting accuracy gains of 4.22 AP on HIBER and 11.02 AP on MMVR over prior state-of-the-art.

What This Paper Is About

Indoor radar systems can detect people without cameras, but they rely on two separate radar views (a horizontal/depth-horizontal projection and a vertical/depth-vertical projection) that must be linked together to recover a full 3D position. Prior methods associate these views implicitly — pairing proposals or letting attention queries find features on their own — which produces ambiguous matches and weak detection in cluttered rooms.

REXO attacks this problem by moving the entire detection process out of the 2D image plane and into the 3D radar volume, so that noisy candidate boxes themselves can be projected onto both views to fetch matching features. The result is a simpler, more robust cross-view association that also lets the authors inject physical priors about where people stand.

Key Contributions

1. 2D-to-3D lifting with explicit cross-view association. Rather than denoising 2D boxes on a horizontal radar image (as DiffusionDet does) and then pairing them with fixed-height vertical windows, REXO denoises full 3D boxes. Each noisy box is projected onto every radar view and feature crops are pulled via RoIAlign. This association cost grows linearly with the number of views, whereas proposal-pairing and query-attention schemes grow quadratically.

2. Cross-view radar-conditioned bounding box detection. The associated multi-view radar features directly condition the denoising network at every timestep. The authors state this is the first diffusion model in radar perception conditioned on multi-view radar input.

3. Ground-level constraint. Leveraging the physical fact that an indoor person is in contact with the floor, the vertical center coordinate is fixed to half the box height (c_y = h/2). This removes one degree of freedom from the diffusion parameter space, regularizes the denoising trajectory, and keeps 3D and 2D gradients flowing jointly.

4. Geometry-aware supervision with 2D refinement. Predicted 3D boxes are transformed through a calibrated radar-to-camera rotation and translation, projected to the image plane, and refined by a small learnable module before a combined 3D + 2D box loss is applied.

Main Findings

• Strong gains on unseen environments. On the MMVR dataset's most difficult split (P2S2, entirely unseen rooms), REXO reaches 23.47 AP versus 12.45 AP for the best baseline RETR — the headline +11.02 margin.

• Consistent improvement on HIBER. On the HIBER "WALK" split, REXO scores 25.33 AP, 62.55 AP50, and 15.83 AP75, beating RETR (22.09 / 59.83 / 10.99) across every metric. The abstract cites this as a +4.22 AP margin over state-of-the-art.

• One split favors the baseline. Under the easiest MMVR configuration (P1S1, single subject, same room, random split), RETR edges out REXO on AP and AP50. The IoU histogram analysis shows this is a threshold artifact: RETR has slightly more boxes in the 0.5–0.75 IoU band, while REXO has more high-quality boxes above 0.75 IoU, which is why REXO still wins at AP75.

• Ground-level constraint helps, as expected. Removing it drops AP from 23.47 to 22.67 on MMVR-P2S2 and from 25.33 to 21.11 on HIBER. The authors note the assumption breaks if someone jumps or stands on furniture.

• Both 3D and 2D supervision are necessary. Setting the 3D loss weight to zero collapses performance to 0.98 AP; setting the 2D weight to 0.1 gives 15.55 AP; using both at full weight yields 23.47 AP. Image-plane supervision is required to train the refinement module.

• Robustness to the number of inference boxes. RETR degrades sharply once queries exceed 10 (12.45 → 6.57 → 3.63 AP at 20, 40, 60 queries), while REXO holds up far better (23.00 → 22.32 → 21.94). This stability is inherited from the diffusion formulation.

• More denoising steps give diminishing but real returns. Going from 1 to 10 steps adds 0.78 AP, at a cost of 60 ms (17 FPS) for 1 step versus 483 ms (2 FPS) for 10 steps. The authors recommend 5 steps (255 ms, 4 FPS) as a practical trade-off.

• The horizontal view carries most of the signal. Replacing the full model with horizontal-only input gives 22.75 AP, vertical-only gives 7.18 AP, and both views give 23.47 AP. The vertical view cannot disambiguate azimuth position on its own.

• Known failure mode: subjects at similar depth. When two people are within 20 cm of each other in depth, their reflections overlap in the vertical heatmap and AP falls to 9.93, compared to 23.47 otherwise.

Methodology in Plain English

The pipeline works in four stages.

First, raw radar pulses from horizontal and vertical antenna arrays are converted via FFTs into two 2D heatmaps (range-azimuth and range-elevation), stacked across four consecutive frames for temporal context. A shared ResNet-plus-FPN backbone extracts multi-scale feature maps from each view.

Second, during training, ground-truth 3D boxes are corrupted with Gaussian noise according to a standard diffusion schedule. A crucial step follows: because people stand on the floor, the vertical center of each box is snapped to half its height before anything else happens.

Third, a denoising network takes these noisy 3D boxes and projects each one onto both radar views, producing 2D boxes in the horizontal and vertical planes. RoIAlign then crops a small fixed-resolution feature patch from each view for each box. These two crops are stacked and fed to a box detector that predicts the clean 3D box and a person-versus-background class score. This is the key architectural move — the boxes themselves drive the feature lookup, so the association is explicit and cheap.

Fourth, the predicted 3D box corners are transformed into camera coordinates using calibrated rotation and translation, projected through a pinhole model, and a learnable refinement module corrects the systematic overshoot of this raw projection. Losses are computed in both the radar 3D space and the image 2D space (each combining GIoU and L1 terms) after Hungarian matching.

At inference, boxes are initialized as pure Gaussian noise and denoised over several steps. Each step conditions on fresh cross-view radar features fetched using the current box estimate, so the conditioning improves as the boxes converge.

Why This Matters

Impact on research. This is the first work to move bounding-box diffusion into full 3D radar space and condition it on multi-view radar input. It shows that the linear-vs-quadratic growth argument for cross-view association is not just an efficiency claim but a measurable accuracy win, particularly in unseen environments. It also demonstrates that physical priors (the ground contact constraint) can be baked directly into the diffusion parameterization rather than handled as post-processing.

Real-world applications:

• Elderly care and assisted living monitoring — detecting falls, gait changes, or inactivity in bathrooms and bedrooms without cameras, preserving privacy.
• Smart building occupancy and HVAC control — counting and locating people in offices and meeting rooms through walls or in the dark.
• Search and rescue in smoke-filled or low-visibility environments — radar penetrates smoke and dust where cameras and LiDAR fail.
• Retail and security analytics — footfall and dwell-time measurement with no personally identifiable imagery.

Industry relevance. The authors are from Mitsubishi Electric's research labs (ITC in Japan and MERL in the US), and the code is publicly released, signaling a push toward deployable indoor radar sensing. Because radar sensors are inexpensive and privacy-preserving compared to cameras, this line of work is directly relevant to consumer electronics, building automation, and healthcare monitoring vendors. The reported 4 FPS at a practical 5-step setting is slow for real-time use but workable for many monitoring scenarios.

Future Directions

• Resolving same-depth multi-person ambiguity. The 20 cm depth-difference threshold is a hard limitation. Better vertical-view disambiguation, temporal tracking, or Doppler-based separation are natural remedies.

• Relaxing or generalizing the ground-level constraint. The assumption fails for jumping, climbing, or seated-on-elevated-surface cases. A learned or soft version of the constraint, or a per-scene prior, could extend the method.

• Closing the inference speed gap. 255 ms per frame at the recommended setting is too slow for many interactive applications. Fewer steps, distillation of the denoising trajectory, or DDIM-style accelerated sampling are obvious next steps.

• Extending beyond a single person class and a single room. The detector currently handles two classes (person, background) and has only been validated in indoor single-room settings. Generalization to outdoor or multi-room layouts, and to additional object categories, remains open.

Target Audience

Researchers and engineers working on radar-based perception, indoor human sensing, or diffusion-based detection. It is most valuable to readers already comfortable with modern detection architectures (DETR-style or proposal-based) who want to understand how generative denoising can be lifted into a 3D sensor-native coordinate space. Practitioners building privacy-preserving occupancy or monitoring systems will find the empirical results and the released code directly actionable. Readers with no prior exposure to diffusion models or radar heatmaps will need to consult the cited background work first.

Authors’ abstract

Multi-view indoor radar perception has drawn attention due to its cost-effectiveness and low privacy risks. Existing methods often rely on {implicit} cross-view radar feature association, such as proposal pairing in RFMask or query-to-feature cross-attention in RETR, which can lead to ambiguous feature matches and degraded detection in complex indoor scenes. To address these limitations, we propose \textbf{REXO} (multi-view Radar object dEtection with 3D bounding boX diffusiOn), which lifts the 2D bounding box (BBox) diffusion process of DiffusionDet into the 3D radar space. REXO utilizes these noisy 3D BBoxes to guide an {explicit} cross-view radar feature association, enhancing the cross-view radar-conditioned denoising process. By accounting for prior knowledge that the person is in contact with the ground, REXO reduces the number of diffusion parameters by determining them from this prior. Evaluated on two open indoor radar datasets, our approach surpasses state-of-the-art methods by a margin of +4.22 AP on the HIBER dataset and +11.02 AP on the MMVR dataset. The REXO implementation is available at https://github.com/merlresearch/radar-bbox-diffusion.