Analyzing Shapley Additive Explanations to Understand Anomaly Detection Algorithm Behaviors and Their Complementarity

AI summary

Overview

• Research area: Unsupervised anomaly detection (UAD), ensemble learning, and explainable AI (XAI) — specifically using SHAP explanations as a model-selection signal.
• Technical level: Advanced. The paper assumes familiarity with SHAP values, Pearson correlation, Mantel tests, Jaccard similarity, NDCG, and AUCPR.
• Scope: The paper tests whether the similarity of SHAP explanations between anomaly detectors predicts the similarity of their outputs, and whether explanation-based diversity metrics can be used to build better ensembles — while showing that diversity alone is not enough without strong individual models.

What This Paper Is About

In unsupervised anomaly detection there are no labels, so practitioners cannot simply benchmark algorithms and pick the winner — and the "no free lunch" theorem means no single detector works best everywhere. Ensembling several detectors is the usual remedy, but choosing detectors that genuinely fail on different cases (rather than duplicating each other) is hard, because many algorithms rely on similar decision cues. This paper proposes characterizing detectors through their SHAP feature attributions and measuring pairwise similarity of those attributions, in order to select ensembles that are more complementary.

Key Contributions

1. Explanation-based behavior analysis. An analysis of UAD algorithms based on their SHAP explanations, demonstrating that model behaviors are correlated with output similarity — detectors with similar SHAP profiles tend to produce correlated anomaly scores and overlapping anomaly sets.
2. Explanation metrics vs. output metrics for selection. A comparison between SHAP-based similarity (linear Pearson correlation of SHAP vectors, ρPS, and NDCG-based ranking similarity, ρNDCG) and output-based similarity (ρScores on anomaly scores, and Jaccard similarity J on binary predictions) for selecting diverse detectors.
3. Diversity versus individual performance. An empirical study quantifying the relative importance of diversity versus individual model quality, showing that model quality remains a critical prerequisite for an effective ensemble.
4. A reusable methodology and codebase. A pipeline combining Kernel SHAP, similarity/dissimilarity matrices, Mantel testing, ensemble aggregation, and linear regression on standardized predictors, with source code released on GitHub.

Main Findings

• SHAP similarity metrics agree with each other. The mean Mantel correlation across datasets between ρPS and ρNDCG is r_M = 0.83, confirming that SHAP-based similarity is consistent whether measured by attribution magnitude or by feature-importance ranking. All reported Mantel correlations are significant at p ≤ 0.004.
• Output similarity metrics agree with each other. ρScores and J correlate at r_M = 0.76, which is expected since models with similar score distributions tend to produce similar binary predictions.
• Explanations predict predictions. ρPS and J correlate at r_M = 0.67, and ρPS and ρScores at 0.55, meaning detectors that reason similarly (as captured by SHAP) also tend to flag similar points as anomalous.
• The strength of these links is dataset-dependent. Per-dataset Mantel correlations (Table 2) range widely: for r_M(ρPS, ρScores) the values run from 18 (WB) to 75 (AN); for r_M(ρPS, J) from −4 (VE) to 84 (LY). Datasets such as BR, LY and YE show r_M(ρPS, ρScores) << r_M(ρPS, J).
• Two clusters of detectors emerge visually. COPOD and ECOD consistently group together (both rely on similar distributional assumptions), while a larger group comprising OCSVM, AutoEncoder, IForest, PCA, KNN and CBLOF exhibits high correlation, sharing distance-metric and data-encoding characteristics.
• Rank aggregation is the best way to combine scores. Evaluated over all 364 ensembles of 3 models drawn from the 14-model pool, rank aggregation gave the best average AUCPR on 11 of 16 datasets, with a mean AUCPR of 52 (×10⁻²) versus 35 for Max and 24 for Mean.
• SHAP-based diversity beats output-based diversity. On 11 of 16 datasets, using ρPS or ρNDCG as the diversity criterion led to better ensemble results than using ρScores or J.
• Diversity is not always beneficial. Diversity–performance correlations are sometimes negative (for example, ρPS reaches −66 on VO and −44 on VE), which the authors attribute to datasets dominated by a single strong model, or to datasets so complex that all models perform poorly.
• Individual performance dominates diversity. In the linear regression predicting ensemble gain from standardized mean individual performance and diversity, the individual performance weight is generally dominant; the diversity coefficient is positive in 12 of 16 datasets, and only for WB and LY does diversity rival or exceed individual performance (ratios of 1.2 and 0.8 respectively).
• Predictors were not collinear. Variance Inflation Factor values across all 16 datasets ranged from 1.0 to 2.3, well below the conventional thresholds of 5 or 10.

Methodology in Plain English

The researchers assembled a pool of 14 unsupervised anomaly detectors (COF, KNN, LOF, IForest, PCA, CBLOF, LODA, HBOS, MCD, OCSVM, DAGMM, DeepSVDD, COPOD, ECOD) implemented through the PyOD library, keeping each model's default hyperparameters and default anomaly threshold, and giving no model any hint about the contamination rate of any dataset.

They ran all 14 detectors on 16 datasets drawn from ADBench — the 50% smallest datasets, with any dataset having more than 20 features removed to keep computation feasible. Each dataset was split five times into 80% training and 20% test sets, with the random seed set to the iteration index for reproducibility.

For every model, they computed explanations with the model-agnostic Kernel SHAP implementation from the Python shap package, using a k-means summary of the background data with k = 50 centroids to stabilize the approximation. Each model therefore yields an n × d matrix of feature attributions.

They then built four pairwise similarity matrices per dataset: two from SHAP (average per-instance Pearson correlation of attribution vectors, ρPS; and a symmetric NDCG-based agreement in ranked feature importance, ρNDCG), one from anomaly scores (ρScores), and one from the Jaccard overlap of the sets of points each model labels anomalous. Each similarity matrix was converted to a dissimilarity matrix via 1 − P and compared using the Mantel test, which correlates the upper triangular elements of two matrices with permutation-based significance testing.

To study complementarity, they enumerated all 364 ensembles of size 3 from the 14 models, scored them with AUCPR (chosen because of dataset class imbalance), and correlated ensemble performance against the diversity of the constituent models under each of the four dissimilarity matrices. Finally, they fit a linear regression predicting ensemble performance gain from standardized mean individual performance and diversity, and checked for multicollinearity with VIF.

Why This Matters

This work shifts model selection for unsupervised anomaly detection away from a purely output-based view toward an explanation-based one. It provides evidence that SHAP attributions carry information about detector behavior that raw scores and Jaccard overlap do not, and it sets a clear boundary condition on the popular "diversity is good" heuristic: diversity helps, but only among models that are individually competent.

• Industrial monitoring and predictive maintenance, where labeled faults are rare or expensive to produce and where ensembles must cover many failure modes.
• Healthcare screening and medical anomaly detection, where datasets such as mammography, thyroid and breast-cancer datasets in this study mirror the low-prevalence, unlabeled regimes practitioners face.
• Safety-critical systems such as the nuclear industry explicitly cited by the authors, where detection must operate without waiting for faults to occur.
• Fraud and document/record anomaly screening, where the diversity of anomaly types means any single detector's assumptions about "normal" are likely incomplete.

For industry, the practical takeaway is a selection procedure: run a pool of detectors, explain them, measure explanation divergence, and prefer ensembles whose members reason differently — while filtering out models that are simply weak.

Future Directions

• Investigating the divergence between SHAP-based similarities and raw-output similarities to refine model selection, since the two metric families highlight different kinds of diversity.
• Integrating methods that characterize individual model performance into the pipeline, given that individual quality proved to be the deciding factor.
• Extending the strategy to time-series unsupervised anomaly detection, where the authors argue ensemble methods can be even more powerful.
• Reducing the computational cost of SHAP — which the authors identify as the main limitation — through approximation techniques or surrogate models, or by substituting faster, still model-agnostic explanation methods such as aggregated Partial Dependence Profiles.

Target Audience

Researchers and graduate students working on anomaly detection, ensemble learning, or explainable AI, as well as practitioners and data scientists who must select detectors for unlabeled industrial or clinical data. Readers need a working knowledge of SHAP, correlation-based similarity, and standard anomaly detection benchmarks to get the most from the paper; the conceptual message about diversity versus individual quality is accessible to a broader audience.

Authors’ abstract

Unsupervised anomaly detection is a challenging problem due to the diversity of data distributions and the lack of labels. Ensemble methods are often adopted to mitigate these challenges by combining multiple detectors, which can reduce individual biases and increase robustness. Yet building an ensemble that is genuinely complementary remains challenging, since many detectors rely on similar decision cues and end up producing redundant anomaly scores. As a result, the potential of ensemble learning is often limited by the difficulty of identifying models that truly capture different types of irregularities. To address this, we propose a methodology for characterizing anomaly detectors through their decision mechanisms. Using SHapley Additive exPlanations, we quantify how each model attributes importance to input features, and we use these attribution profiles to measure similarity between detectors. We show that detectors with similar explanations tend to produce correlated anomaly scores and identify largely overlapping anomalies. Conversely, explanation divergence reliably indicates complementary detection behavior. Our results demonstrate that explanation-driven metrics offer a different criterion than raw outputs for selecting models in an ensemble. However, we also demonstrate that diversity alone is insufficient; high individual model performance remains a prerequisite for effective ensembles. By explicitly targeting explanation diversity while maintaining model quality, we are able to construct ensembles that are more diverse, more complementary, and ultimately more effective for unsupervised anomaly detection.