Distribution-Based Feature Attribution for Explaining the Predictions of Any Classifier

AI summary

Overview

Research area: Explainable AI (XAI), specifically model-agnostic feature attribution for post-hoc explanations of black-box classifiers.

Technical level: Advanced. The paper combines a formal problem definition with kernel density estimation, Shapley-value theory, and a large empirical study; readers will benefit from familiarity with probability distributions and attribution metrics.

Scope: The paper proposes a formal definition of feature attribution, uses it to critique existing model-agnostic methods, and introduces a distribution-based explainer called DFAX, validated on ten real-world datasets against five state-of-the-art baselines plus a random baseline. An extended version accepted for oral presentation at the 40th AAAI Conference on Artificial Intelligence (AAAI-26).

What This Paper Is About

Feature attribution methods assign importance scores to input features to explain why a classifier made a particular prediction, but the field has never agreed on a formal definition of what such an explanation should be. The authors argue that an explanation is only valid if it is supported by the probability distribution the classifier actually operates on — empirically represented by the unmodified dataset — and that many popular methods break this rule by constructing synthetic or out-of-distribution instances. Their goal is to supply that missing definition and then build a method, DFAX, that satisfies it while also using the entire dataset rather than only a local slice of it.

Key Contributions

1. A formal problem definition. The paper introduces Definition 1, which states that the explanatory model and its scores must be built from the unmodified dataset X, an empirical representation of the underlying distribution P, and that any modification of X that changes the distribution invalidates the attribution.

2. A criterion-based analysis of existing methods. Using Definition 1, the authors classify prior model-agnostic methods, showing that local approximation methods such as LIME and perturbation-based implementations using Shapley sampling values violate the criterion, while DLIME, SLISE, MAPLE, and Shapley regression values comply but carry other limitations.

3. DFAX (Distributional Feature Attribution eXplanations). A new model-agnostic method (Definition 2) that scores a feature by the difference between the conditional probability of the target instance given the target class and given all other classes, computed with a one-dimensional kernel density estimator per feature. The authors state it is the first feature attribution approach based on this principle and the first explainer to approach attribution by directly leveraging the underlying data distribution.

4. Extensive quantitative and qualitative evaluation. Experiments on ten datasets with five baselines plus a random sanity check, covering deletion and insertion scores, sentiment attribution on movie reviews, tissue-type prediction in spatial transcriptomics, and a runtime comparison.

Main Findings

• DFAX leads on both metrics. Averaged across the ten datasets, DFAX with a Gaussian kernel (DFAX_G) achieves a deletion score of .3244 and an insertion score of .7708, versus .3344 and .7470 for the SiNNE-based variant (DFAX_S). The best baseline on deletion is DLIME at .4595; the best baseline on insertion is DLIME at .6612. The random baseline scores .5709 deletion and .5838 insertion.

• Rankings confirm the gap. Average rankings are 1.5 (DFAX_G) and 1.6 (DFAX_S) on deletion, and 1.2 and 2.0 on insertion. The next-best method, DLIME, ranks 4.1 and 3.8. DFAX secures first or second rank on nine of the ten datasets; the only exception is Diabetes, where the top three methods perform comparably.

• Two baselines fail the sanity check. The average insertion scores of SHAP (.5463) and MAPLE (.5800) fall below the random baseline (.5838).

• Gaussian kernel generally beats SiNNE within DFAX. DFAX_G shows better results than DFAX_S overall, especially on insertion score, although the paper reports that the final hyperparameters differed per dataset for both variants.

• Compliance does not guarantee quality. LINEX and the SHAP implementation used here do not satisfy Definition 1, while DLIME, SLISE, and MAPLE do — yet the compliant baselines still underperform because they do not fully exploit the information in X. DLIME, SLISE, and MAPLE are limited to local regions drawn from a subset of X.

• Qualitative sentiment attribution. On RottenTomatoes snippets, DFAX highlights "compelling", "fascinating", and "moving" for positive sentiment and "bad", "dull", and "too" for negative sentiment. DLIME instead selects "real" over "compelling" in one positive example and "humor" over "bad" in one negative example, which the authors describe as unreasonable.

• Spatial transcriptomics results. On the HER2st dataset (527 cells, 314 genes, 6 classes, Random Forest accuracy .73), masking the 157 non-salient genes identified by DFAX preserves 95.64% of the original predictions, compared with 79.51% for DLIME. DFAX produces 6 misclassifications of cancer cells versus 44 for DLIME.

• Runtime. DFAX achieves the shortest runtime on all three representative datasets used in the comparison and is often faster than the baselines by orders of magnitude. The paper presents these results in a log-scale figure; per-method runtimes in seconds are not enumerated in the text.

Methodology in Plain English

The authors begin conceptually: they define what a legitimate feature attribution is. Their key stipulation is that the explanation must be derived from the dataset exactly as it is, with no synthetic instances or mixtures of the target instance and background data. Otherwise the explanation describes a distribution the classifier was never trained or qualified to handle.

They then audit existing methods against this rule. LIME-style local approximation fits a surrogate on randomly sampled neighbors that do not come from the data distribution. The common SHAP implementation replaces feature subsets of dataset points with values from the target instance, creating hybrid points that are not independent and identically distributed samples from P. Shapley regression values do obey the rule, but require retraining the classifier on every feature subset, which the paper calls computationally infeasible.

DFAX itself is simple. For each feature taken alone, it estimates two one-dimensional probability densities: how likely the target instance's value is among dataset points of the predicted class, and how likely it is among points of all other classes. The attribution score is the first minus the second, so a feature gets a high score when its value is characteristic of the predicted class and uncharacteristic of the rest. Density is estimated with a kernel density estimator — Gaussian by default, or the faster SiNNE substitute.

Because the density for each feature depends only on the dataset and not on the classifier at query time, DFAX is fully decoupled from the model: once predictions on X are computed, the classifier is never queried again. And if the kernel admits a finite-dimensional feature map, the kernel mean map over X can be precomputed once, making later attributions roughly constant-time after that one-off cost.

Why This Matters

Impact on research. The paper argues that the absence of a formal problem definition is the root cause of many limitations in existing attribution methods, because methods were designed and evaluated without a shared standard. Definition 1 is offered as a guideline for both evaluating and designing attribution methods, and the analysis reclassifies well-known methods by whether they meet it.

Real-world applications:

• Oncology and spatial transcriptomics: identifying which of 314 genes are most critical to determining a cell's tissue type, as demonstrated on HER2-positive breast tumor data.
• Sentiment and text analysis: locating the words in a movie review that drive a classifier's sentiment decision, as shown on RottenTomatoes snippets.
• Healthcare and clinical risk models: the Diabetes dataset (520 instances, 16 features) and the Bankruptcy dataset (6,819 instances, 95 features) illustrate tabular decision settings where knowing which variable drove a prediction matters.
• Privacy- or cost-constrained deployments: because DFAX never queries the classifier after an initial pass over X, it suits settings where the model is expensive to call, proprietary, or inaccessible, or where the goal is to explain the data's inherent class structure using ground-truth labels instead of predictions.

Industry relevance. The authors emphasize efficiency: DFAX reports the shortest runtime on all three tested datasets, often by orders of magnitude, which speaks to practical deployment at scale. The decoupling from the classifier is also attractive to organizations that cannot expose model internals.

Future Directions

• Axiomatic properties of DFAX. The authors state they plan to investigate what formal guarantees or axioms the method satisfies — a natural complement to the Shapley-based theoretical grounding of competing methods.
• Feature-group attribution. Extending DFAX from individual features to groups of features is listed as planned work, which would matter for grouped inputs such as the 314-gene panel used in the HER2st study.
• Compliant and practical baselines. The paper notes that variants of LINEX and SHAP exist that would satisfy Definition 1 but carry other restrictions; exploring those trade-offs is an open comparison.
• Generalizing the definition's reach. Since Definition 1 was proposed as a guideline for any feature attribution method, an open question is how existing and future methods can be redesigned to meet it without inheriting the local-focus or computational limitations identified here.

Target Audience

XAI researchers and graduate students working on post-hoc explanation methods, particularly those interested in formal problem definitions and distributional approaches to attribution. Practitioners who need to explain black-box classifiers in regulated or high-stakes domains — clinical, financial, and scientific — will find the model-agnostic framing, the efficiency claims, and the decoupling from the classifier most relevant. Readers who want implementation details will need the appendix, which specifies hyperparameter search spaces and per-dataset final values for DFAX and the baselines.

Authors’ abstract

The proliferation of complex, black-box AI models has intensified the need for techniques that can explain their decisions. Feature attribution methods have become a popular solution for providing post-hoc explanations, yet the field has historically lacked a formal problem definition. This paper addresses this gap by introducing a formal definition for the problem of feature attribution, which stipulates that explanations be supported by an underlying probability distribution represented by the given dataset. Our analysis reveals that many existing model-agnostic methods fail to meet this criterion, while even those that do often possess other limitations. To overcome these challenges, we propose Distributional Feature Attribution eXplanations (DFAX), a novel, model-agnostic method for feature attribution. DFAX is the first feature attribution method to explain classifier predictions directly based on the data distribution. We show through extensive experiments that DFAX is more effective and efficient than state-of-the-art baselines.