CHiQPM: Calibrated Hierarchical Interpretable Image Classification

AI summary

Overview

• Research area: Interpretable machine learning (interpretability-by-design image classification), with connections to conformal prediction and human-AI complementarity.
• Technical level: Advanced. The method builds on quadratic programming for feature selection, conformal prediction theory, and fine-tuning of deep vision backbones.
• Scope in one sentence: The paper introduces CHiQPM, a by-design interpretable image classifier that produces more contrastive global class representations and sample-specific hierarchical local explanations, and that can be calibrated to output interpretable prediction sets with conformal guarantees.

What This Paper Is About

Globally interpretable models explain their general behavior, but a single-image decision still needs a local explanation, and heatmap-style local explanations do not communicate certainty. CHiQPM extends the QPM line of work (compact, contrastive, general features assigned to classes) by enforcing that more pairs of classes share nearly identical feature representations, by producing hierarchical local explanations, and by turning those explanations into a conformal prediction (CP) mechanism. The goal is a model that keeps close to non-interpretable accuracy while giving global explanations, local explanations, and calibrated set predictions in one system.

Key Contributions

1. Calibrated Hierarchical QPM (CHiQPM): A model built on a heavily constrained discrete quadratic problem (QP) that selects features from a black-box model and assigns them to classes; the features then adapt to that optimal solution, yielding a model that is interpretable both globally and locally.
2. Novel hierarchical local explanations plus an interpretable conformal predictor: CHiQPM constructs sample-specific hierarchies, can be calibrated to reach a target coverage by ascending dynamically constructed class hierarchies and selecting the appropriate level, and is described by the authors as the first model with a built-in interpretable set prediction calibratable via CP.
3. Feature Grounding Loss (L_feat): Combined with an additional ReLU, this loss leads to more grounded and sparser features, facilitating compact hierarchical explanations along more human concepts.
4. State-of-the-art evaluation across architectures and datasets: CHiQPM is evaluated as a point predictor and as a built-in interpretable calibrated coherent set predictor on CUB-2011, Stanford Cars and ImageNet-1K, where the gap to the black-box baseline is reported to be more than halved.

Main Findings

• Point-prediction accuracy: CHiQPM attains state-of-the-art accuracy among compact point predictors. In Table 1, CHiQPM scores 85.3 on CUB, 91.9 on Stanford Cars and 75.3 on ImageNet-1K, compared with QPM at 85.1 / 91.8 / 74.2 and the Dense Resnet50 baseline at 86.6 / 92.1 / 76.1. The abstract states CHiQPM maintains 99% accuracy of non-interpretable models.
• Contrastiveness: CHiQPM scores 99.9 on CUB, 100 on Stanford Cars and 99.9 on ImageNet-1K, the highest in Table 1, ahead of QPM (96.0 / 97.7 / 89.3) and PIP-Net (99.5 / 99.5 / not reported).
• Structural Grounding: CHiQPM reaches 75.0, versus 47.9 for QPM, 29.2 for SLDD-Model, 23.4 for Q-SENN, 13.9 for ProtoPool, 6.7 for PIP-Net, 34.0 for Dense Resnet50 and 2.5 for glm-saga 5.
• Contrastive pairs are rare in prior work: On CUB-2011, contrastive representations are described as fairly rare, with on average just 0.13% pairs per class.
• Set-prediction efficiency: Table 2 reports that CHiQPM's built-in set predictor is competitive with CP baselines. At CUB α = 0.12 CHiQPM averages 1.22 predicted classes versus 1.16 for THR and 6.30 for APS; at CUB α = 0.05 it averages 9.05 versus 2.41 for THR and 11.3 for APS; at ImageNet α = 0.22 it averages 1.10 versus 1.05 for THR and 16.7 for APS. All reported methods are described as very close to or reaching the desired coverage.
• Nonconformity score variants: The paper's s = s_sel and s = s_up variants produce larger sets than the final CHiQPM configuration across the reported α values on all three datasets.
• Gradient imbalance motivating L_feat: In the toy example of Figure 5 (3 classes, 7 features), at the average activation on the CUB dataset the Ground Truth Exclusive (GTE) feature has a roughly 4000 times higher gradient than the other assigned features, which are shared with the similar class.
• Hyperparameters: Defaults are n_wc = 5 features per class, n_f* = 50 features total, density parameter ρ = 0.5 (enforcing on average one very similar class each) and λ_feat = 3, since higher values cause reduced accuracy.

Methodology in Plain English

The pipeline follows QPM. First, a dense "black box" model is trained with an auxiliary Feature Diversity Loss (L_div) so that feature maps activate on distinct image locations. Second, a discrete quadratic problem jointly selects a small set of features and assigns them to classes, using a class-feature similarity matrix A (built from the dense model), a feature-feature similarity matrix R or bias term b to favour distinct local features, and constraints forcing exactly n_f* selected features and exactly n_wc features per class. CHiQPM adds a hierarchical constraint: a set K of the most similar class pairs is computed from the class-class similarity K = A Aᵀ − I, with threshold θ set at the (2·ρ·n_c)-th highest value, and pairs in K are constrained to share n_wc − 1 features. The initial global solution is then relaxed to find the best way to satisfy |P| ≥ |K|. Finally, features are fine-tuned with W* fixed, using the new Feature Grounding Loss alongside a ReLU so the model "only reasons positively" and negligible activations are suppressed. L_feat measures the difference in average activation between the ground truth class's features and all other features, scaled by the maximum activation so activations are not inflated.

For set prediction, activations are sorted per class to order the assigned features by strength, forming a sample-specific tree. Predicting at a fixed depth n means predicting every class that shares the same top n features as the argmax class. Calibration uses split conformal prediction with a nonconformity score built from the activations along the shared path (s_up), extended to account for the activation at the point of diversion (s_sel, for subtree selection), and limited to a maximum level n_limit that still reaches the target coverage (α).

Why This Matters

• Research impact: The paper argues that it is the first work to propose the notion of interpretable CP, and that the hierarchical constraint increases the number of classes that can be contrastively explained globally, a property the authors measure and compare against models such as PIP-Net, ProtoPool, SLDD-Model, Q-SENN and QPM.
• Real-world applications (as framed or motivated in the paper):
  • Medicine, where local explanations support a human expert present during inference.
  • Autonomous driving, where no expert is present and global interpretability allows reasoning to be tested and verified before deployment.
  • Science, which the paper names alongside medicine as a domain that can profit from additional interpretability.
  • Any regulated application where interpretability is legally required, which the paper cites as motivation.
• Industry relevance: The work is a collaboration between the Institute for Information Processing / L3S at Leibniz Universität Hannover and Intel Labs (Germany and USA), and the code is released at https://github.com/ThomasNorr/CHiQPM/.

Future Directions

• Balancing point versus set prediction: The paper notes that increasing ρ raises the risk of pairing classes that do not share n_wc − 1 general concepts, which harms performance, creating a tradeoff that it says should be set using calibration data; how best to tune this for different domains remains open.
• Extending to new domains and backbones: The method is demonstrated on CUB-2011, Stanford Cars and ImageNet-1K with Resnet50, Resnet34, Inception-v3 and Swin-Transformer-Small; scaling behavior beyond these settings is left to further study.
• Conditional coverage: The paper contrasts unconditional coverage (which it targets efficiently) with conditional coverage, which aims for the desired expected accuracy on all test samples, including difficult ones; the authors frame the broader interpretable-CP direction as newly opened.
• Interpretability measurement: The paper notes that measuring interpretability is an unsolved task and cites polysemantic neurons as an obstacle to measuring grounding, leaving room for better metrics.

Target Audience

Researchers and practitioners in interpretable and trustworthy machine learning, conformal prediction, and safety-critical computer vision (for example medical imaging, autonomous driving and scientific discovery) who need models whose decisions can be inspected globally, explained locally per sample, and accompanied by calibrated prediction sets. Readers should be comfortable with deep learning training pipelines, quadratic optimization formulations and the split conformal prediction framework.

Note: the paper content supplied for this summary is truncated partway through Section 4.2 (the sentence about contrastiveness continues beyond the provided text), so any conclusion, limitations or further results that follow in the full paper are not reported here.

Authors’ abstract

Globally interpretable models are a promising approach for trustworthy AI in safety-critical domains. Alongside global explanations, detailed local explanations are a crucial complement to effectively support human experts during inference. This work proposes the Calibrated Hierarchical QPM (CHiQPM) which offers uniquely comprehensive global and local interpretability, paving the way for human-AI complementarity. CHiQPM achieves superior global interpretability by contrastively explaining the majority of classes and offers novel hierarchical explanations that are more similar to how humans reason and can be traversed to offer a built-in interpretable Conformal prediction (CP) method. Our comprehensive evaluation shows that CHiQPM achieves state-of-the-art accuracy as a point predictor, maintaining 99% accuracy of non-interpretable models. This demonstrates a substantial improvement, where interpretability is incorporated without sacrificing overall accuracy. Furthermore, its calibrated set prediction is competitively efficient to other CP methods, while providing interpretable predictions of coherent sets along its hierarchical explanation.