Layer-Wise Modality Decomposition for Interpretable Multimodal Sensor Fusion

AI summary

Overview

• Research area: Explainable AI (XAI) for multimodal perception in autonomous driving, specifically post-hoc attribution in camera–radar, camera–LiDAR, and camera–radar–LiDAR sensor-fusion models.
• Technical level: Advanced. The paper builds on Layer-Wise Relevance Propagation (LRP) and Deep Taylor Decomposition (DTD), and assumes familiarity with Jacobians, local linearization, and normalization-layer internals.
• Scope (one sentence): The paper introduces Layer-Wise Modality Decomposition (LMD), a post-hoc, model-agnostic method that splits a pretrained fusion model's prediction into per-modality and bias components at every layer, and validates it on the nuScenes benchmark.

What This Paper Is About

When a perception model fuses camera, radar, and LiDAR inputs, the sensor information becomes entangled inside the network, so it is hard to tell how much each modality contributed to a given prediction. The authors propose LMD, which decomposes the prediction of a pretrained fusion model into separate camera, radar (or LiDAR), and bias terms across all layers, without changing the original architecture or its performance. They claim this is the first approach to attribute the predictions of a sensor-fusion perception model in autonomous driving to its individual input modalities.

Key Contributions

1. LMD itself: a novel post-hoc interpretability method that decomposes each sensor modality's contribution across all layers of a complex multimodal model, operating without any modification to the original architecture.
2. A theoretical framework: derivations showing that, after linearizing activation and normalization layers, every layer output equals the sum of modality-specific features and a bias term, satisfying both the original-function constraint and a "separation property" (each modality feature is unaffected by the others).
3. New evaluation metrics: perturbation-based metrics using Pearson Correlation and Mean Squared Error to measure whether a modality-specific prediction responds to changes in its own input and stays stable when other modalities are perturbed.
4. A study of LMD variants: several bias-splitting strategies (identity, uniform, ratio rules) for BatchNorm and LayerNorm, quantitatively compared from a modality-separation perspective.

Main Findings

• Modality separation is achieved: With the ratio rule, when the radar input is replaced, the radar-based output shows a Pearson correlation of 0.05 with its original output and a mean squared error of 9.05, while the camera-based output remains unaffected with a correlation of 1.00. In the LiDAR + Camera setting, the corresponding values are 0.09 and 16.17 for the perturbed modality, with a correlation of 1.00 for the unperturbed one.
• Baselines stay entangled: Baseline models show residual correlation across modalities (for example, 0.22 ± 0.10 for R_p/R and 0.76 ± 0.07 for R_p/C in Radar + Camera), indicating predictions remain entangled and fail to achieve modality separation.
• Both linearization components are needed: Ablations show that linearizing only activations or only normalization leaves cross-modality correlation. In Radar + Camera, applying activation linearization alone gives R_p/R of 0.56 ± 0.09; normalization alone gives 0.50 ± 0.08; applying both gives 0.05 ± 0.02.
• The ratio rule matters: Among the four tested variant combinations (uniform–identity, identity–identity, identity–uniform, identity–ratio), the identity rule for BatchNorm combined with the ratio rule for LayerNorm produces the clearest separation — lowest perturbed-modality correlation (0.05 for R_p/R and 0.15 for C_p/C in Radar + Camera) with full stability (1.00) and zero MSE for the fixed modality. Other configurations yield higher cross-modality correlations.
• The bias component is not noise: Visualization of the bias term shows it "exhibits a certain degree of perceptual capability," containing constant effects and high-order interactions largely originating from activation-layer linearization.
• Cheaper than alternatives: LMD requires two forward passes with O(1) computational complexity and O(1) memory, versus LRP's 2 forward + 1 backward pass at O(√N_l) memory and Shapley-based methods' O(2^M) forward passes and O(2^M) complexity for M modalities.
• Complementary to SHAP: Applying SHAP on top of LMD improves cross-modality consistency under modality replacement compared with SHAP alone — for example, in Radar + Camera, R_p/C rises from 0.6909 ± 0.11 to 0.9385 ± 0.04 and C_p/R from 0.6746 ± 0.07 to 0.8942 ± 0.05. In the three-modality setting, C_p L_p/R goes from 0.7066 ± 0.08 to 0.9574 ± 0.01.

Methodology in Plain English

The authors take a pretrained fusion model and make it explainable without retraining it. Their starting point is the idea that a network's prediction can be broken into pieces using a first-order Taylor expansion, where separate terms correspond to each input modality. That only works cleanly if the network is linear, so they run the model once on the full multimodal input and record how the non-linear parts behave — for ReLU activations, a binary mask of which neurons were active, and for normalization layers, the statistics needed to freeze them. They then rerun the model with those non-linear layers replaced by their linearized versions, which preserves the original output exactly while making the layers additive. Because the resulting network is linear, they can feed it camera-only, radar-only (or LiDAR-only), and zero-input cases and get the model's prediction split exactly into modality-specific parts plus a bias part. They test this on SimpleBEV fusion models on the nuScenes benchmark, and evaluate the decomposition by swapping in an uncorrelated sample for one modality: a good decomposition should make that modality's prediction change a lot, while leaving the other modalities' predictions untouched. They also compare variants of how the bias term in normalization layers is handled (distributed uniformly, dropped entirely, or handled by the ratio rule).

Why This Matters

• Research impact: The paper opens a direction for post-hoc interpretability in multimodal perception, where no standard evaluation for modality decomposition previously existed. It provides a theoretical decomposition guarantee plus new perturbation-based metrics that future work can reuse, and its code is publicly released.
• Real-world applications:
  • Diagnosing autonomous-driving perception failures by identifying which sensor drove a wrong or right detection.
  • Safety validation and regulatory auditing of multi-sensor perception stacks.
  • Sensor-placement and sensor-suite design, by revealing which modalities actually carry the signal for given scenarios.
  • Sensor-failure analysis, since the decomposition can show whether a prediction depends on a modality that is degraded or missing.
• Industry relevance: For autonomous vehicle developers, LMD offers a way to inspect a production-scale fusion model without retraining it or sacrificing accuracy, at the low cost of two forward passes — compared with exponential cost for Shapley-based attribution or an extra backward pass and higher memory for LRP.

Future Directions

• Extending LMD beyond the two-modality derivation used for clarity — the paper provides an M-modality extension in Appendix F and reports quantitative three-modality results in Appendix D.1, but the main-text analysis centers on two-modality settings.
• Further study of the bias term, which the authors show carries real perceptual capability and high-order interactions from activation linearization, but which they do not interpret in detail.
• Deeper integration with other attribution methods — the LMD + SHAP combination is evaluated in one table, and the paper notes the results are summarized in a table that is truncated in this content.
• Applying the theoretical framework to other multimodal architectures beyond SimpleBEV, since the authors state LMD is generalizable and readily applicable to other multimodal models.

Target Audience

Researchers and practitioners in explainable AI and autonomous-driving perception who are already familiar with LRP/DTD-style attribution and multimodal fusion architectures. It is also useful for engineers at companies building or auditing sensor-fusion perception systems, and for graduate students looking for a worked example of turning a theoretical decomposition into a practical, benchmark-validated interpretability tool. Readers without a background in backpropagation-based attribution or normalization-layer mechanics will find the method sections demanding.

Authors’ abstract

In autonomous driving, transparency in the decision-making of perception models is critical, as even a single misperception can be catastrophic. Yet with multi-sensor inputs, it is difficult to determine how each modality contributes to a prediction because sensor information becomes entangled within the fusion network. We introduce Layer-Wise Modality Decomposition (LMD), a post-hoc, model-agnostic interpretability method that disentangles modality-specific information across all layers of a pretrained fusion model. To our knowledge, LMD is the first approach to attribute the predictions of a perception model to individual input modalities in a sensor-fusion system for autonomous driving. We evaluate LMD on pretrained fusion models under camera-radar, camera-LiDAR, and camera-radar-LiDAR settings for autonomous driving. Its effectiveness is validated using structured perturbation-based metrics and modality-wise visual decompositions, demonstrating practical applicability to interpreting high-capacity multimodal architectures. Code is available at https://github.com/detxter-jvb/Layer-Wise-Modality-Decomposition.