Does Explainability Transfer? A Controlled Benchmark of Attribution Methods on Vision Transformers and CNNs

AI summary

Overview

Research area: Explainable AI (XAI) for computer vision, specifically post-hoc attribution methods evaluated across CNN and Vision Transformer (ViT) architectures.

Technical level: Intermediate. The paper assumes familiarity with CNNs, self-attention, and basic attribution concepts, but its central question — whether explanation quality transfers across architectures — is stated in accessible terms.

Scope: A single controlled benchmark that holds data, target selection, preprocessing, and metric configuration constant while varying 13 attribution methods across 8 backbones and scoring them on 5 dimensions of explanation quality.

What This Paper Is About

Almost everything known about how well XAI attribution methods perform was established on convolutional neural networks (CNNs), yet Vision Transformers now dominate computer vision and process information through fundamentally different mechanisms (discrete patch tokens, self-attention, class tokens). The paper asks whether rankings and conclusions drawn from CNNs carry over to ViTs and the newer hierarchical, hybrid, and linear-attention variants. It answers this by building a standardized evaluation harness that compares attribution methods across the whole architecture family under one fixed protocol.

Key Contributions

1. Controlled cross-architecture benchmark. A unified framework that holds data, target selection, preprocessing, and experimental settings constant to systematically compare attribution methods across CNNs, isotropic ViTs, and advanced hierarchical/efficient transformer backbones.

2. Multi-axis evaluation suite. 13 attribution methods from four algorithmic families (gradient-based, CAM-based, attention-native, perturbation-based) scored on five dimensions: faithfulness (Faithfulness Correlation, Faithfulness Estimate), localization (Pointing Game, Energy-Based Pointing Game), robustness (Max-Sensitivity), complexity (Sparseness), and computational cost (wall-clock time per explanation).

3. Architectural vulnerability analysis. Theoretical and empirical identification of architecture-specific failure modes, including Grad-CAM++ collapse under global self-attention in ViTs and degradation of gradient-based methods caused by local window partitioning in Swin Transformers.

4. Open-source evaluation framework. A registry-based benchmarking harness with configuration files and per-sample outputs, released at https://github.com/Nishan-Charlie/VIT_XAI_Bench.

Main Findings

• Attribution performance is strongly architecture-dependent. Rankings established on CNNs do not reliably transfer to transformer-based models, and the paper's central claim is that attribution rankings shift with architecture.

• CAM-based methods lead under bounding-box localization — but the metric saturates. Grad-CAM and Grad-CAM++ achieve the highest Pointing Game scores on CNNs and most ViTs. Grad-CAM scores 0.98 on ResNet-50, 0.93 on ViT-B/16, and 1.00 on Swin-B, PVT-v2 (Grad-CAM), and MobileViT. The authors show the Pointing Game has a high floor: the mean bounding-box area fraction is 0.59, close to the empirical 0.61 random-point baseline, so a random peak already lands inside the box most of the time.

• Under a pixel-perfect dense metric, CAM dominance largely disappears. Using the Energy-Based Pointing Game (EBPG), which measures the fraction of attribution energy inside the pixel-perfect ImageNet-S silhouette, the "perfect" 1.00 cells fall to about 0.5, no method exceeds 0.60, and the gradient method VarGrad becomes competitive or better (VarGrad: 0.58 on ResNet-50, 0.55 on MobileViT, 0.57 on EfficientViT-B2). The EBPG random-map baseline is the mean mask area, 0.18.

• CAM methods fail badly on linear-attention architectures. Grad-CAM Pointing Game drops to 0.70 at the 9M-parameter EfficientViT-B1 (Grad-CAM++ 0.44) and 0.55 at the 24M-parameter EfficientViT-B2 (Grad-CAM++ 0.56), far below CNN and most ViT results and, under the dense metric, near the random floor (Grad-CAM EBPG 0.39 and 0.27).

• CAM methods show limited robustness on global-attention transformers, while attention rollout is stable but poorly localized. Attention Rollout provides consistently stable explanations with poor localization (Pointing Game 0.64 on ViT-B/16). The robustness table (Max-Sensitivity) appears in Table 5, which falls beyond the truncated content provided.

• Faithfulness correlation offers limited discrimination between methods. The authors highlight this as evidence against single-metric evaluation. (The per-image dispersion analysis of Faithfulness Correlation is referenced to a section not included in the truncated content.)

• Attention-native methods have a structural applicability gap. Attention Rollout and AttnLRP are defined only on the isotropic ViT-B/16, which exposes a global class-token attention matrix — meaning they are undefined on six of the seven ViT variants evaluated. The released AttnLRP toolbox has no bespoke rules for windowed (Swin), spatial-reduction (PVT-v2), multi-axis (MaxViT), convolution-hybrid (MobileViT), or linear cross-covariance (EfficientViT) attention.

• No prior benchmark spans this coverage. Compared with ROAR, CLEVR-XAI, Rao et al., Bodria et al., FunnyBirds, and Wu et al., the paper states it is the first to cover CNN, isotropic ViT, and modern ViT backbones on real data with all four method families and all five evaluation axes under a single fixed protocol.

Methodology in Plain English

The researchers built a single, rigid test harness and ran everything through it so that differences in results could only come from architecture, not from experimental setup.

• Data: A fixed subset of 1,000 validation images from ImageNet-S, which provides dense semantic segmentation masks on top of ImageNet-1k. For the coarse Pointing Game, each dense mask is reduced to its tight axis-aligned bounding box.

• Models: Eight pretrained backbones spanning the ViT family's evolution — ResNet-50 (CNN), ViT-B/16 (isotropic ViT), Swin-B (shifted-window attention), PVT-v2-B2 (spatial-reduction attention), MaxViT-S (multi-axis attention), MobileViT-v2 (convolution-transformer hybrid), and EfficientViT-B1/B2 (multi-scale linear attention). These range roughly from 9M to 88M parameters and are matched by tier rather than exact parameter count, so the design does not cleanly separate architecture from model capacity.

• How explanations are produced: Each image passes through a pretrained classifier; the predicted class is the explanation target. Each applicable attribution method produces a heatmap. Because gradient methods produce signed values and CAM outputs are positive, all heatmaps go through one shared normalization: absolute value of relevance scores, then min-max scaling to [0, 1].

• Fairness controls: CAM methods target each backbone's canonical layer — the terminal feature map returned by forward_features (a 7×7 grid for the hierarchical, multi-axis, and linear-attention backbones), and the last transformer block's input normalization for the isotropic ViT, because its head pools only the class token. No layer search is performed, so the comparison reflects each backbone's standard configuration.

• Implementation: 224×224 inputs, PyTorch 2.6, a single NVIDIA GPU, standard ImageNet normalization statistics. Gradient and perturbation methods use Captum, CAM variants use pytorch-grad-cam, attention-native methods use custom forward hooks, metrics use Quantus 0.6, and all backbones are public timm 1.0 checkpoints. Stochastic methods use a fixed seed of 0.

• Metric budgets: Faithfulness Correlation uses 20 perturbation runs per image with feature subsets of size 224 (reduced from the Quantus default of 100 to keep the full 13×8 grid tractable); Faithfulness Estimate steps and occludes 448 features per iteration; Max-Sensitivity uses 3 localized perturbation samples per image.

• Statistical testing: Bonferroni-corrected Friedman tests with Nemenyi post-hoc comparisons.

Why This Matters

Impact on research. The paper challenges a widely held assumption that attribution quality conclusions from CNN studies carry over to modern vision backbones. It shows that a single popular metric (the bounding-box Pointing Game) can create the illusion of CAM superiority through metric saturation, and that faithfulness correlation alone barely discriminates between methods. The work argues for architecture-aware, multi-dimensional evaluation as the standard, and it documents a concrete tooling gap: attention-native XAI methods have no defined rules for the attention mechanisms that now dominate the ViT family.

Real-world applications. The paper itself does not enumerate specific application domains beyond noting that deep learning is used in "scientific, industrial, and safety-critical domains" where decisions "require inspection, verification, and technical justification." Any downstream use would depend on the domain in which explanations are deployed.

Industry relevance. Teams that select an attribution method based on CNN-era benchmarks and then deploy it on a Swin, MaxViT, MobileViT, or EfficientViT model may be relying on explanations that do not localize the object — the EfficientViT Grad-CAM results (Pointing Game 0.70 and 0.55, EBPG 0.39 and 0.27) are the clearest example. The released registry-based framework gives practitioners a reproducible way to check explanations on their own backbone.

Future Directions

• Disentangling architecture from scale. The authors state that the 0.70-versus-0.55 gap between the two EfficientViT sizes may include a capacity component the design cannot isolate, and leave an iso-parameter sweep to future work.

• Sweeping the faithfulness configuration. Because the Faithfulness Correlation conclusion is specific to the reduced configuration (20 runs, subset size 224, single seed), the authors call for a sweep over subset size and run count.

• Extending attention-native rules. The paper identifies the absence of AttnLRP rules for windowed, spatial-reduction, multi-axis, convolution-hybrid, and linear cross-covariance attention as an open gap that currently makes those cells undefined rather than merely poor.

• Broadening the metric suite. The paper explicitly frames the work as a step toward architecture-aware, multi-dimensional evaluation, raising the question of which additional metrics would discriminate between methods where faithfulness correlation does not.

Target Audience

Researchers and practitioners in explainable AI and computer vision who select, implement, or evaluate attribution methods — particularly those applying CNN-era XAI tools to transformer backbones, and those designing benchmarks or relying on the Pointing Game and faithfulness correlation as evidence of explanation quality.

Authors’ abstract

Most evidence on the effectiveness of explainable artificial intelligence (XAI) attribution methods has been established on convolutional neural networks (CNNs), with limited investigation into whether these conclusions generalize to the diverse Vision Transformer (ViT) architectures that now dominate computer vision. This paper presents a controlled benchmark that evaluates attribution quality across five dimensions: faithfulness, localization, robustness, complexity, and computational cost. A standardized framework assesses 13 attribution methods from four algorithmic families on eight representative backbones spanning CNNs, isotropic ViTs, hierarchical transformers, hybrid architectures, and linear-attention transformers. The results show that attribution performance is strongly architecture-dependent and that rankings established on CNNs do not reliably transfer to transformer-based models. CAM-based methods achieve the highest scores under the conventional bounding-box localization metric on CNNs and most ViTs but perform poorly on linear-attention architectures. Pixel-level dense-mask evaluation further reveals that these gains largely reflect metric saturation rather than accurate localization. CAM-based methods also exhibit limited robustness on global-attention transformers, whereas attention rollout provides consistently stable explanations with poor localization. Furthermore, faithfulness correlation offers limited discrimination between attribution methods, highlighting the limitations of single-metric evaluation. These findings challenge prevailing conclusions on attribution performance and demonstrate the need for architecture-aware, multi-dimensional evaluation. The open-source code for the evaluation framework and benchmark results is available at https://github.com/Nishan-Charlie/VIT_XAI_Bench.