Zero-Shot Textual Explanations via Translating Decision-Critical Features

AI summary

Overview

• Research area: Interpretability of image classifiers — specifically zero-shot generation of natural-language explanations for the decisions of pretrained CNN and Transformer classifiers.
• Technical level: Advanced. The paper assumes familiarity with Integrated Gradients, CLIP-style vision–language alignment, feature visualization, and sparse autoencoders.
• Scope: The paper introduces TEXTER, a three-stage framework that isolates neurons contributing to a classifier's prediction, visualizes them as "concept images," and translates those concept images into CLIP space to retrieve textual explanations.

What This Paper Is About

Existing zero-shot methods for explaining image classifiers align whole images with language, so their descriptions report what is visible (dominant objects, backgrounds) rather than what actually drove the prediction. TEXTER addresses this by first isolating the decision-critical features — the features encoded in the neurons that contribute most to the predicted class — and only then aligning them to a vision–language model. The goal is textual explanations that faithfully reflect a classifier's reasoning without concept annotations, retraining, or paired explanation–label data.

Key Contributions

1. TEXTER itself. A zero-shot framework that translates decision-critical features of a classifier into the CLIP feature space to produce textual explanations, rather than aligning global image features as prior work does.
2. Cross-architecture generalizability via a sparse autoencoder. The authors show that a TopK SAE is not essential for CNNs but is crucial for Transformers, whose compositional, entangled feature spaces make it hard to isolate decision-critical factors; the SAE enables TEXTER to work across both model families.
3. Quantitative and qualitative validation. Concept images are assessed for whether they preserve the information used in the original prediction, and generated explanations are compared against Random and Text-To-Concept baselines on semantics-based metrics across five classifiers.
4. Class-conditioned explanation. TEXTER can retrieve explanations for an arbitrary target class, enabling class-wise analysis of an image — including comparison of the predicted class against the ground-truth class in a misclassification case.

Main Findings

• Concept images preserve decision-critical information — for CNNs even without the SAE. On 1,000 randomly selected ImageNet-1K test images (200 classes, 5 images per class), ResNet-18 and ResNet-50 reach Acc@1 of 0.92 without the SAE, and DINO ResNet-50 reaches 0.79.
• Transformers collapse without the SAE and recover with it. ViT goes from Acc@1 0.11 and Cos 0.11 without the SAE to Acc@1 0.99 and Cos 0.44 with it; DINO ViT-S/8 goes from 0.08 to 0.89 in Acc@1 and from 0.34 to 0.61 in Cos. The authors attribute the raw Transformer failure to the compositional and entangled nature of its feature space.
• The SAE can slightly reduce CNN validity. ResNet-18 drops from Acc@1 0.92 to 0.80, ResNet-50 from 0.92 to 0.83, and DINO ResNet-50 from 0.79 to 0.48, though the authors describe the concept images as remaining largely comparable.
• TEXTER beats baselines on all semantics-based metrics. On 100 randomly selected PASCAL VOC test images using the top-scoring class per image, TEXTER achieves higher CLIP-Score, lower LPIPS (AlexNet and SqueezeNet), and higher Feature Similarity than both Random and Text-To-Concept for every model tested — for example, CLIP-Score 0.2340 versus 0.2172 (Text-To-Concept) and 0.2065 (Random) on DINO ResNet-50.
• Global-feature methods describe the wrong content. In the qualitative comparison for a person prediction, Text-To-Concept focuses on background elements such as sedan shape, while TEXTER identifies hairstyle, hair, and facial expression; for TV monitor, Text-To-Concept highlights furniture features such as light sensor, while TEXTER identifies high-definition display, flat screen, and tv stand.
• Explanation style varies by architecture. For Bedlington terrier, ResNet-50 and ViT focus on texture attributes such as silver-gray coat, whereas DINO ViT-S/8 attends to structural facial features such as a sheep-like face and narrow muzzle; for Bee eater, ViT and DINO ViT-S/8 retrieve relational concepts such as pair bonding behavior and mating display.
• Class-conditioned analysis clarifies misclassifications. For an image labeled water snake but predicted as stick insect, TEXTER with target class stick insect produces slender antennae and slender body segments, suggesting twig-like background fragments are misread as insect limbs, while with target class water snake it produces slithering through vegetation and venomous look.

Methodology in Plain English

TEXTER has three stages.

1. Concept image generation. For a given input and predicted class, the method scores every neuron in a feature vector using Integrated Gradients (baseline set to the zero vector, M = 100 steps) and keeps the top k_neu = 6. It then runs feature visualization — specifically magnitude-constrained optimization (MACO), which optimizes the Fourier phase spectrum while holding magnitude constant — to synthesize an image that maximizes the sum of activations of those selected neurons. The result is a "concept image" that emphasizes the internal evidence behind the prediction. A TopK sparse autoencoder (embedding dimension 8× the original feature vector, Top-K ratio of 10%, trained with batch size 1024, learning rate 5×10⁻⁴, Adam, 10 epochs) is optionally inserted so the neuron selection happens in a sparse, more interpretable space.
2. Vision–language alignment. An affine layer h(f(x)) = Wf(x) + b is trained to map the classifier's feature space into the CLIP vision feature space, minimizing squared error over a training set — following the same protocol and data as Text-To-Concept, using 20% of the ImageNet-1K training set — with the ViT-B/16 vision encoder.
3. Explanation generation. The concept image is passed through the classifier, then through the aligner, and the result is compared by cosine similarity against the text embeddings of candidate short phrases in a concept bank. The top k_con = 3 are returned as explanations. The bank is built by prompting an LLM (GPT-3.5-turbo) for generic class-level concepts and a VLM (Qwen2.5-VL-7B-Instruct) for image-grounded concepts, yielding 100 + 30 = 130 descriptions per class by default.

Only the SAE modules and the aligner are trainable. They require no additional annotations, are not part of the classifier's inference path, and therefore do not affect the original accuracy.

Why This Matters

• Research impact: The paper reframes zero-shot textual explanation as a neuron-isolation problem followed by alignment, rather than a global feature-alignment problem. It also supplies concrete evidence that SAEs matter much more for Transformer-based classifiers than for CNNs, which is a reusable empirical finding for interpretability research.
• Real-world applications:
  • Debugging deployed image classifiers by surfacing the evidence behind a specific prediction, including spurious cues such as background fragments.
  • Auditing misclassifications in multi-label settings, where an image contains several objects and the model must be checked against each predicted class.
  • Reviewing model behavior class by class when classes share overlapping visual cues, using TEXTER's class-conditioned retrieval.
  • Providing human-readable rationales in domains where stakeholders need to judge whether a model relies on meaningful patterns.
• Industry relevance: Because TEXTER requires no concept annotations or explanation–label pairs and does not retrain the original classifier, it can be applied post hoc to existing pretrained models. The authors are explicit that the extra SAE/aligner training is offline and one-time, not per-image, but they state they do not claim efficiency and flag scalability to larger concept banks and higher-resolution settings as a limitation.

Future Directions

1. Better concept banks. The authors note that explanation quality is bounded by the concept bank's coverage: missing or under-represented concepts cannot be retrieved, and the number of descriptions per class trades off coverage against noise. They suggest low retrieval confidence could serve as a signal to expand or adapt the bank, and that TEXTER is agnostic to how the bank is built.
2. Robust faithfulness protocols. The paper states that semantics-based metrics (CLIP-Score, LPIPS, Feature Similarity) capture semantic consistency but not causal faithfulness, and that the perturbation-based test using CLIPSeg masks assumes evidence is well localizable and separable by masking — which can break for fine-grained or distributed cues, and may be insensitive when candidate descriptions overlap heavily.
3. Efficiency and scale. The authors explicitly decline to claim efficiency and list scalability to larger concept banks and higher-resolution settings as open.
4. Robustness under distribution shift. TEXTER may be dataset-dependent like other post-hoc methods; mismatched concept banks or auxiliary modules can degrade retrieval and concept-image validity, which the authors propose treating as a warning signal.

Target Audience

Researchers and practitioners in explainable AI and computer vision interpretability who work with pretrained image classifiers and vision–language models, especially those already familiar with concept-based explanations (TCAV, concept bottleneck models), zero-shot alignment methods such as Text-To-Concept and ZSNLE, and sparse autoencoders. It is also relevant to engineers who need post-hoc, annotation-free explanations for existing CNN or Transformer classifiers without retraining them.

Authors’ abstract

Textual explanations make image classifier decisions transparent by describing the prediction rationale in natural language. Large vision-language models can generate captions but are designed for general visual understanding, not classifier-specific reasoning. Existing zero-shot explanation methods align global image features with language, producing descriptions of what is visible rather than what drives the prediction. We propose TEXTER, which overcomes this limitation by isolating decision-critical features before alignment. TEXTER identifies the neurons contributing to the prediction and emphasizes the features encoded in those neurons -- i.e., the decision-critical features. It then maps these emphasized features into the CLIP feature space to retrieve textual explanations that reflect the model's reasoning. A sparse autoencoder further improves interpretability, particularly for Transformer architectures. Extensive experiments show that TEXTER provides more faithful and interpretable explanations than existing methods. The code is available at \url{https://github.com/tttt-0814/TEXTER}.