Rethinking Saliency Maps: A Cognitive Human Aligned Taxonomy and Evaluation Framework for Explanations

AI summary

Overview

Research area: Explainable AI (XAI) for computer vision, specifically the taxonomy and evaluation of saliency-map explanations.

Technical level: Intermediate. The paper is conceptual and metric-driven rather than mathematically dense; familiarity with saliency methods (Grad-CAM, Integrated Gradients, attention rollout) and basic classification terminology helps.

Scope: One sentence — the paper proposes a two-axis taxonomy (Reference-Frame × Granularity, "RFxG") for organizing saliency explanations by user intent, and introduces four new perturbation-based faithfulness metrics to evaluate explanations along both axes.

What This Paper Is About

Saliency maps are widely used to show which image regions drove a model's prediction, but there is no consensus on what question a given map is supposed to answer, and existing evaluation metrics almost all measure only pointwise faithfulness to a single target class. This paper argues that explanations should instead be organized by the user's actual query — pointwise ("Why Husky?") versus contrastive ("Why Husky and not Shih-tzu?"), and fine-grained class level versus coarse-grained group level ("Why Husky?" versus "Why Dog?"). The goal is to supply both a conceptual structure and practical metrics so that explanations can be evaluated for alignment with human intent, not just model behavior.

Key Contributions

1. The RFxG taxonomy. A dual-axis framework for saliency explanations: a reference-frame axis separating pointwise from contrastive explanations, and a granularity axis separating fine-grained class-level from coarse-grained group-level interpretations. The paper notes explanations can also sit between the points, e.g. class-group contrastive questions like "Why Husky and not other Dogs?"

2. Four new faithfulness metrics. Contrastive Contrastivity Score (CCS), Class Group Contrastivity (CGC), Pointwise Group Score (PGS), and Contrastive Group Score (CGS), all grounded in structured perturbation and score comparison. The authors state these complement rather than replace standard metrics, and that they avoid the thresholding and multiplication operations used in prior contrastive metrics.

3. A group-level labeling of ImageNet classes derived from the WordNet hierarchy. Candidate superordinate concepts containing at least five subordinate leaf nodes were used; for example, sports car, cab, and limousine were grouped under car. Where no existing group was found, classes were merged with neighbors until at least five classes were included. The resulting dataset of image-class-group triplets is to be released publicly as a benchmark.

4. A large evaluation framework. Ten saliency methods, four model architectures, and three datasets, with code released at https://github.com/yonisGit/RFxG.

Main Findings

• IIA leads across the board. Iterated Integrated Attributions (IIA) consistently outperforms the other methods across all datasets and model architectures, particularly on the contrastive-class metric CCS and the group-level metrics PGS and CGS. The authors attribute this to IIA's integration of attributions over multiple intermediate network layers, which captures features at several semantic scales.

• Group-level explanations score higher than class-level ones. PGS and CGS values are consistently higher than CGC and CCS across nearly all methods and models. The paper reads this as evidence that faithful explanations for group-level concepts are easier to produce than for individual class distinctions within a semantically coherent group.

• Class-vs-class beats class-vs-group. CCS scores exceed CGC scores, implying that contrasting a class against a specific alternative is easier than contrasting it against its entire group, possibly because discriminative power is diluted across many similar classes.

• TAttr is the strongest Transformer method. Transformer Attribution ranks consistently second after IIA across all metrics, ahead of Rollout and GCV.

• IG underperforms everywhere. Integrated Gradients performs worst across all metrics, especially in contrastive settings, producing coarse, diffuse maps that highlight many non-discriminative regions.

• Existing contrastive metrics misrank focused maps. IIA ranks only third on CDROP despite its focused visual behavior, which the authors argue shows that density normalization and thresholding in metrics such as CAUC and CDROP penalize sparse attributions. They also note CAUC multiplies prediction scores, hurting interpretability, and depends on a thresholding mechanism that assumes many zero-valued pixels — an assumption often violated by dense transformer-based and localized maps.

• Statistical significance. The reported quantitative comparative results are stated to be statistically significant by a t-test with a p-value of 0.05.

Methodology in Plain English

The authors define a classifier's softmax probability for a class, then take a saliency map and binarize it into a "top-α" mask marking the most salient pixels. They suppress those pixels (multiplied out to black), producing a perturbed image, and observe how the model's probabilities change. Perturbation proceeds in 10% steps from α=0.1 to α=0.9, and the Area Under the Curve (AUC) aggregates the results over the whole trajectory. All scores use softmax probabilities rather than raw logits. Black-pixel masking is the default; Gaussian blur, uniform noise baseline, and other alternatives suggested by Sturmfels et al. (2020) were also tried, and trends stayed consistent.

Each of the four metrics applies that same perturbation machinery to a different question. CCS measures the AUC of the probability gap between two classes as salient regions are removed. CGC averages the confidence rise of competing members of a semantic group together with the drop in the target class. PGS averages the confidence drop across all classes in a group. CGS measures the drop in one group's confidence alongside the rise in another group's.

Experiments used the validation sets of PASCAL VOC, ImageNet ILSVRC 2012, and COCO 2017, with the highest-scoring classes treated as ground truth. For class A they used the model's top predicted class; class B was the second-highest-scoring class from the same semantic group as A. Groups came from the WordNet hierarchy. Four models were tested — ResNet-50, ConvNext-Base, ViT-Base, and ViT-Small — with five CNN methods (Grad-CAM, Integrated Gradients, Score-CAM, SHAP, IIA) and five Transformer methods (Grad-CAM-ViT, Attention Rollout, GAE, TAttr, IIA). Where methods lacked native contrastive or group support, the authors adapted them, for example using Contrastive-GradCAM and a minimally modified Contrastive-SHAP, and computing explanations with respect to the score difference f_A − f_B for class-contrastive settings. Experiments ran on an NVIDIA DGX 8xA100 server using PyTorch. The exact number of images in each evaluation is not reported in the provided content.

Why This Matters

The paper argues that most saliency research is model-centered, optimizing visualizations without regard to the question a user actually asked, and that this ambiguity undermines trust, interpretability, and usability of explanation methods. By making user intent explicit and measurable, the framework offers a way to compare explanation methods on the questions they were never designed to answer.

Real-world applications the work points toward:

• Medical imaging, where the paper states higher-level categories carry more practical relevance than fine-grained single-class labels.
• Autonomous driving, cited alongside medical imaging as a domain where semantic abstraction matters more than narrow class distinctions.
• Model debugging and bias detection, since faithfulness underpins user trust, bias detection, and effective debugging.
• Benchmarking and method selection, via the released image-class-group triplet dataset and the RFxG metric suite for comparing explanation methods under a consistent protocol.

For industry, the practical relevance is a shift from "does this heatmap look right?" to "does this heatmap answer the question the user asked?" — with a repeatable metric suite and public code that teams can apply when choosing or auditing explanation methods.

Future Directions

• Extending the taxonomy to excluded method families. The experiments deliberately focus on saliency methods producing a single map per question and are agnostic to model internals; concept-based and generative explanation techniques are excluded and remain open.
• Closing the gap between contrastive metrics. The discrepancy where IIA ranks only third on CDROP while leading on CCS and CGC raises the question of how contrastive faithfulness should be measured without density normalization and thresholding artifacts.
• Broadening group-level explanation research. The strong PGS and CGS results are framed as evidence that high-quality group-based explanations are feasible, which the authors present as a promising direction for letting users query semantic abstractions.
• Limitations and further work. The authors state that limitations and future directions are discussed in the appendix (Section F), whose content is not included in the provided material.

Target Audience

XAI and interpretability researchers, especially those working on saliency methods and evaluation metrics; computer vision practitioners who need to justify model decisions to non-expert stakeholders; and HCI-oriented researchers studying how the framing of an explanation question changes what counts as a good explanation. Reviewers and benchmark designers who need a structured protocol for comparing explanation methods across contrastive and group-level settings will also find the metric suite and released benchmark directly useful.

Authors’ abstract

Saliency maps are widely used for visual explanations in deep learning, but a fundamental lack of consensus persists regarding their intended purpose and alignment with diverse user queries. This ambiguity hinders the effective evaluation and practical utility of explanation methods. We address this gap by introducing the Reference-Frame $\times$ Granularity (RFxG) taxonomy, a principled conceptual framework that organizes saliency explanations along two essential axes:Reference-Frame: Distinguishing between pointwise ("Why this prediction?") and contrastive ("Why this and not an alternative?") explanations. Granularity: Ranging from fine-grained class-level (e.g., "Why Husky?") to coarse-grained group-level (e.g., "Why Dog?") interpretations. Using the RFxG lens, we demonstrate critical limitations in existing evaluation metrics, which overwhelmingly prioritize pointwise faithfulness while neglecting contrastive reasoning and semantic granularity. To systematically assess explanation quality across both RFxG dimensions, we propose four novel faithfulness metrics. Our comprehensive evaluation framework applies these metrics to ten state-of-the-art saliency methods, four model architectures, and three datasets. By advocating a shift toward user-intent-driven evaluation, our work provides both the conceptual foundation and the practical tools necessary to develop visual explanations that are not only faithful to the underlying model behavior but are also meaningfully aligned with the complexity of human understanding and inquiry.