The effect of whitening on explanation performance

AI summary

Overview

Research area: Explainable Artificial Intelligence (XAI) — specifically the reliability of feature attribution methods, and the role of data preprocessing (whitening) in shaping explanation quality.

Technical level: Intermediate. The paper is readable for someone with basic familiarity with machine learning classifiers and the idea of feature importance, but the theoretical section involves covariance matrices, eigendecompositions, and Bayes-optimal linear classifiers.

Scope (one sentence): The paper empirically and theoretically tests whether five linear whitening transforms can reduce the tendency of sixteen feature attribution methods to assign importance to non-informative "suppressor" variables, using the XAI-TRIS synthetic benchmark and a two-dimensional linear model.

What This Paper Is About

Feature attribution methods often highlight variables that carry no real class information but are statistically correlated with features that do. These are called suppressor variables, and prior work (Haufe et al., 2014; Wilming et al., 2022; Wilming et al., 2023) showed they cause systematic misinterpretations. Because suppression arises from dependencies between features, the authors ask whether whitening — a standard decorrelation preprocessing step — can repair explanation correctness, and under what conditions.

Key Contributions

1. An empirical evaluation of 16 feature attribution methods combined with 5 distinct whitening transforms on the XAI-TRIS benchmark, covering four binary image classification scenarios (LIN, MULT, RIGID, XOR) across two background types (WHITE, CORR) and three model architectures (LLR, MLP, CNN).
2. A demonstration that whitening's benefit is not uniform: it varies substantially across whitening technique, XAI method, and model architecture.
3. A theoretical analysis using the two-dimensional linear generative model of Wilming et al. (2023), deriving closed-form Bayes-optimal weight vectors after each whitening transform to test analytically whether the suppressor's influence can be removed.
4. The finding that in the two-dimensional case only partial regression annihilates the suppressor weight, and an explanation of why that result does not generalize to higher dimensions or to non-linear settings.

Main Findings

• Correlated background hurts explanations: Precision of XAI methods declines on the CORR background relative to the uncorrelated WHITE background, supporting the claim that correlated noise induces suppressor variables that degrade attribution fidelity.

• Whitening partially rescues performance: Applying whitening to the CORR data restores explanation performance to varying degrees, with some techniques approaching the WHITE ("baseline") results on both the EMD and precision metrics.

• Three transforms perform best: Sphering, symmetric orthogonalization, and optimal signal preservation (OSP) whitening produce the highest EMD values and the best precision, with symmetric orthogonalization and OSP yielding near-identical results — consistent with the fact that they differ mainly in which matrix (overlap matrix versus correlation matrix) is diagonalized.

• Two transforms fall short: Cholesky whitening and partial regression sometimes improve on the unwhitened correlated case but remain clearly behind the other techniques.

• Cholesky's ordering dependence is a disadvantage: Cholesky whitening introduces a sequential ordering (top-left pixel unchanged, then row-by-row left-to-right orthogonalization under the (H, W) convention); only one of N! possible orderings was tested, an arbitrary choice the authors identify as a likely cause of its weaker performance.

• Partial regression's limited decorrelation: Partial regression regresses each feature against all others and replaces it with residuals; it does not guarantee uncorrelated, unit-variance features, which the authors suggest explains its weaker empirical showing.

• Qualitative pattern shift: Importance maps move from diffuse to more localized patterns after whitening; symmetric orthogonalization produced the most concentrated maps, most closely mirroring the ground truth. Gradient-based methods such as Gradient SHAP and Integrated Gradients visibly sharpen from dispersed to concentrated attributions as whitening is applied.

• Theory confirms no general protection: In the two-dimensional linear model (with c = 0.8, s1² = 1.0, s2² = 1.0 shown in the figure), only partial regression yields w2 = 0, i.e., removes the suppressor weight. Sphering, symmetric orthogonalization, OSP, and Cholesky all retain non-zero weight on the suppressor feature x2.

• The scope condition for whitening succeeding: A general protective effect could only be expected if the observed features are linear combinations of at most as many independent underlying signal or noise factors as there are features. When there are more underlying signals than features, whitening must mix discriminative and non-discriminative components into new features, which can worsen explanations.

• Inconsistency across methods: Supplementary analyses across all 16 XAI methods and 4 model-ignorant baselines show improvement in many cases with whitening, but the improvements are not consistent across all XAI methods.

• Numerical scores not reported in the source text: The exact precision and EMD values underlying these trends are presented in figures; no specific numeric values are reported in the text provided.

Methodology in Plain English

The authors start from data with a known correct answer. XAI-TRIS generates 8×8 pixel images containing tetromino shapes (T-shaped versus L-shaped) overlaid on noise. Two background types are used: WHITE (independent Gaussian noise) and CORR (the same noise smoothed with a Gaussian filter with spatial standard deviation σ_smooth = 3, which creates correlations between neighboring pixels and therefore suppressor variables). Each dataset has N = 10000 samples and dimensionality D = 64. The four scenarios are LIN (fixed-position tetrominoes, additive, linear), MULT (tetrominoes multiplied with the background, non-linear), RIGID (tetrominoes randomly translated and rotated, additive), and XOR (both tetrominoes present, classification by XOR configuration).

They train three models — a linear logistic regression (single fully connected layer), an MLP with four ReLU-activated fully connected layers, and a CNN with four ReLU-activated convolutional layers plus max-pooling — each ending in a two-neuron softmax output. Every model must reach at least 80% test accuracy so comparisons are fair.

They then apply whitening to the CORR data (the WHITE data serves as an unwhitened baseline). Each of the five whitening methods is a linear transform derived from a matrix decomposition: sphering (sample covariance matrix), symmetric orthogonalization (overlap/second-moment matrix), OSP (correlation matrix), Cholesky (Cholesky decomposition of the covariance matrix, giving a lower-triangular transform), and partial regression (residuals from regressing each feature on all others). Data are centered, and the covariance matrix is regularized for numerical stability by comparing the smallest eigenvalue to a threshold of 1×10⁻¹⁶.

Attribution methods — with LIME, LRP, Gradient SHAP, and Integrated Gradients discussed in the main text, and 16 methods total in the appendices — produce importance scores for each input pixel. These are compared to the ground truth (the tetromino pixels) using two metrics: precision, which looks at how many of the top-k ranked features are truly important, and earth mover's distance (EMD), computed with the algorithm of Bonneel et al. (2011) as implemented in the Python Optimal Transport library (Flamary et al., 2021), using Euclidean pixel distance as the ground metric and normalized so that 1.0 is a perfect explanation. Four model-ignorant baselines (Sobel filter, Laplace filter, uniform random samples, and the rectified test sample itself) establish whether XAI methods actually beat approaches that ignore the model.

Finally, they take Wilming et al.'s (2023) two-dimensional generative model — x = az + η with y = z, a = (1, 0)ᵀ, Z ~ Rademacher(1/2), and correlated noise parameterized by standard deviations s1, s2 and correlation coefficient c ∈ [−1, 1] — and analytically derive the Bayes-optimal weight vector w = Σ⁻¹(Wμ1 − Wμ2) after each whitening transform W, to check whether the non-zero weight on the suppressor x2 can be eliminated.

Why This Matters

Research impact. The paper tests a plausible and widely used remedy (whitening) against a specific, theoretically grounded failure mode (suppressor-induced false attributions). It shows that whitening is not a general fix, and it connects empirical XAI benchmark results to closed-form theory, giving the field a clearer boundary condition: whitening helps when the number of underlying signal or noise factors does not exceed the number of features, and is unlikely to help otherwise. It also reinforces the standing warning that linear model weights should not be read as importance scores.

Real-world applications.

• Medical diagnosis support, where a model might use correlated nuisance signals (for example, scanner-specific artifacts) alongside genuine symptoms, and clinicians only look at the top-ranked features.
• Scientific discovery pipelines using high-dimensional correlated sensor data, where suppressors can generate false scientific conclusions.
• Image classification with systematic background variation such as lighting conditions, where sparse explainability is used for auditing.
• Regulatory review of machine learning models, where attribution outputs may be used as evidence of what a model relies on.

Industry relevance. Preprocessing choices such as whitening are already standard for numerical stability and training speed. This paper quantifies a side effect of those choices on interpretability tooling, which matters to teams shipping models with explanation dashboards: changing a preprocessing step can silently change what an explanation says, and the direction of that change depends on both the XAI method and the model architecture.

Future Directions

1. Developing preprocessing methods that move beyond linear decorrelation to capture and remove non-linear or higher-order redundancy, and measuring how those affect explanation performance in the presence of suppressors.
2. Investigating whether XAI methods themselves need dedicated mechanisms to separate true predictors from correlated suppressors, rather than relying on preprocessing alone.
3. Exploring Cholesky whitening's dependence on feature ordering — the paper tested one ordering out of N! possibilities, and the impact of other orderings is an open question.
4. Extending the theoretical analysis to higher-dimensional settings, since partial regression's suppressor-removal property holds in the two-dimensional case but depends on the linear combination of all other features, which will not hold as dimensionality increases.

Target Audience

XAI researchers and practitioners who select or evaluate feature attribution methods; machine learning engineers who control preprocessing pipelines and explanation tooling; and applied scientists in domains such as neuroimaging, medicine, and remote sensing where correlated nuisance variables are common and model interpretations drive real decisions. The theoretical section is best suited to readers comfortable with linear algebra and statistical decision theory.

Authors’ abstract

Explainable Artificial Intelligence (XAI) aims to provide transparent insights into machine learning models, yet the reliability of many feature attribution methods remains a critical challenge. Prior research (Haufe et al., 2014; Wilming et al., 2022, 2023) has demonstrated that these methods often erroneously assign significant importance to non-informative variables, such as suppressor variables, leading to fundamental misinterpretations. Since statistical suppression is induced by feature dependencies, this study investigates whether data whitening, a common preprocessing technique for decorrelation, can mitigate such errors. Using the established XAI-TRIS benchmark (Clark et al., 2024b), which offers synthetic ground-truth data and quantitative measures of explanation correctness, we empirically evaluate 16 popular feature attribution methods applied in combination with 5 distinct whitening transforms. Additionally, we analyze a minimal linear two-dimensional classification problem (Wilming et al., 2023) to theoretically assess whether whitening can remove the impact of suppressor features from Bayes-optimal models. Our results indicate that, while specific whitening techniques can improve explanation performance, the degree of improvement varies substantially across XAI methods and model architectures. These findings highlight the complex relationship between data non-linearities, preprocessing quality, and attribution fidelity, underscoring the vital role of pre-processing techniques in enhancing model interpretability.