Catalyst: Out-of-Distribution Detection via Elastic Scaling

AI summary

Overview

Research area: Out-of-distribution (OOD) detection for deep neural networks — specifically post-hoc scoring methods applied to already-trained image classifiers (Computer Vision).

Technical level: Intermediate. The core idea is simple, but the paper assumes familiarity with logits, penultimate feature vectors, global average pooling (GAP), energy scores, and metrics like FPR95 and AUROC.

Scope: The paper introduces Catalyst, a post-hoc "elastic scaling" framework that multiplies an existing OOD score by an input-dependent factor γ derived from pre-pooling channel statistics, and evaluates it on CIFAR and ImageNet benchmarks across several CNN backbones.

What This Paper Is About

Existing post-hoc OOD detectors (Energy, ReAct, DICE, ASH, SCALE, KNN) all build their scores from either the output logits or the penultimate feature vector produced by global average pooling. The authors argue this reliance creates an information bottleneck, because the raw pre-pooling activation map still contains channel-wise statistics — mean, standard deviation, and maximum activation — that pooling throws away. Catalyst's goal is to recover that discarded signal and use it to sharpen the separation between in-distribution (ID) and OOD samples, without retraining the model.

Key Contributions

1. A complementary, post-hoc framework. Catalyst computes an input-dependent scaling factor γ on the fly from pre-pooling channel-wise statistics and fuses it with an existing baseline score, generalizing across ResNet, DenseNet, and MobileNet architectures.
2. Large and consistent empirical gains. The paper reports average FPR95 reductions of 32.87% on CIFAR-10 (ResNet-18), 27.94% on CIFAR-100 (ResNet-18), and 22.25% on ImageNet (ResNet-50) when Catalyst is combined with ReAct.
3. Ablations and statistical analysis validating the choice of layer (the penultimate pre-pooling map), the fusion strategy (multiplicative), the statistics (mean, standard deviation, max), and the clipping threshold (Appendix B and Section 5).
4. Demonstrated generality beyond logit-based scores, including substantial improvements to the distance-based KNN detector and comparisons against AdaScale, NCI, and fDBD.

Main Findings

• Pre-pooling statistics carry discriminative signal. Figure 1 shows channel-wise mean, standard deviation, maximum activation, and entropy extracted from the penultimate pre-pooling map of an ImageNet-trained ResNet-50, evaluated against Textures as OOD, exhibit ID/OOD separation.
• Catalyst improves Energy alone. On CIFAR-10 (ResNet-18), Energy scores 35.50 FPR95 / 94.17 AUROC, while Catalyst(m) reaches 16.59 FPR95 / 97.10 AUROC. On CIFAR-100 (ResNet-18), Energy is 70.21 FPR95 / 83.54 AUROC versus Catalyst(m) at 45.96 FPR95 / 89.37 AUROC.
• Catalyst + ReAct sets new reported results. On CIFAR-10 (ResNet-18), Catalyst(m)+ReAct reaches 13.19 FPR95 / 97.59 AUROC, compared with ReAct at 29.76 / 95.19 and SCALE at 21.05 / 96.19. On CIFAR-100 (ResNet-18), it reaches 34.66 FPR95 / 91.70 AUROC versus ReAct at 57.76 / 87.97.
• Gains hold on DenseNet-101. CIFAR-10: Catalyst(m)+ReAct gives 8.42 FPR95 / 98.26 AUROC versus ReAct at 17.13 / 96.61. CIFAR-100: 28.06 FPR95 / 93.06 AUROC versus ReAct at 52.89 / 87.18.
• ImageNet-scale results. Averaged over four OOD datasets, Catalyst(m) reduces FPR95 by 44.35%, 47.99%, 43.69%, and 21.23% versus energy using ResNet-34, ResNet-50, MobileNet-v2, and DenseNet-121 respectively. Catalyst(m)+ReAct improves FPR95 by 25.39%, 19.41%, 5.57%, and 12.62% over previous best results on those same four backbones. As one concrete example, ResNet-50 FPR95 moves from SCALE's 21.89 (95.32 AUROC) to Catalyst(m)+ReAct's 17.64 (95.93 AUROC).
• Works with distance-based KNN. On CIFAR-100 (ResNet-18), Catalyst(m) on top of KNN (66.81 FPR95 / 83.40 AUROC) achieves a 43.84% FPR95 reduction, reaching 37.52 FPR95 / 90.99 AUROC. On ImageNet, Catalyst(μ) on a ResNet-50 produces a 52.13% reduction in average FPR95, moving KNN from 64.05 FPR95 / 95.56 AUROC to 31.11 / 98.46.
• Works with DICE. On ImageNet, Catalyst(μ)+DICE improves relative FPR95 by 22.24% (ResNet-50) and 15.41% (MobileNet-v2).
• Outperforms contemporary methods. Against AdaScale's reported DenseNet-101 CIFAR-100 results, Catalyst(m)+ReAct yields a 32.45% gain over the best AdaScale variant; against NCI (OpenOOD settings) it achieves an average FPR95 advantage of 33.43% on CIFAR-10 (ResNet-18); against fDBD it achieves a 65.54% FPR95 reduction on ImageNet (ResNet-50).
• Multiplicative beats additive on robustness. Both fusion strategies reach similar peak performance, but the additive variant requires tuning its clipping threshold at an extremely low percentile (≤ 1st percentile for ResNet-50), making it operationally fragile; the multiplicative threshold sits at a stable, moderate percentile.
• Layer choice matters. γ distributions from early-to-mid residual stages (Layers 1–3) of ResNet-50 overlap heavily between ID and OOD; only the final residual stage (Layer 4), immediately before GAP, separates well.
• Statistic choice matters. Median consistently degrades performance; Shannon entropy is inconsistent — it gave a strong 14.65% improvement in one case (MobileNet-v2 on ImageNet) but minimal gains on other architectures such as ResNet-50. Combining statistics (e.g., mean + std) did not yield significant gains over the best single statistic.
• Near-OOD is weaker. On distinguishing CIFAR-10 from CIFAR-100, Catalyst yields only marginal gains over SCALE, which the authors attribute to the high similarity of learned penultimate representations.
• Negligible overhead. Catalyst(μ) adds less than 0.0001% to a ResNet-50 forward pass; Catalyst(σ), the most complex statistic, adds less than 0.01%. ID classification accuracy is unchanged.

Methodology in Plain English

A trained CNN produces, just before its final classification layer, a stack of activation maps — one per channel, each with spatial extent k×k. Standard practice collapses each channel with global average pooling into a single number, giving the feature vector that most OOD detectors consume.

Catalyst instead keeps the un-pooled map and computes three per-channel statistics: the mean (which is exactly the pooled feature vector), the standard deviation across spatial positions, and the maximum activation. Because OOD inputs can produce extreme activations, each statistic is clipped at a threshold c, set to the p-th percentile of the ID activation distribution (following the same convention as ReAct). The clipped values are summed over channels to produce a single scalar γ for the input.

That scalar is then multiplied by the baseline OOD score — the "elastic scaling" — stretching or shrinking the score depending on the input's statistics. The paper also tested adding γ to the score instead, but chose multiplication because it proved far less sensitive to the clipping percentile. A one-dimensional threshold on the resulting score then decides ID versus OOD, calibrated so that 95% of ID data is correctly classified. Because nothing in training changes, Catalyst is plug-and-play onto Energy, ReAct, DICE, SCALE, or KNN. Notably, the authors deliberately avoid assuming an OOD validation set: the clipping percentile is tuned on a proxy OOD set built by adding pixel-wise Gaussian noise to ID validation images.

Why This Matters

This paper argues that a widely shared design assumption in OOD detection — that the pooled feature vector is the right and sufficient input — leaves a free, essentially zero-cost signal on the table. If the reported gains hold up, OOD detectors could be improved by a small change to an existing pipeline rather than a new training procedure or architecture.

Real-world applications, as framed by the paper:

• Medical diagnosis, where the paper states erroneous predictions can have severe consequences.
• Autonomous driving, cited as another safety-critical setting.
• Any deployed classifier that will encounter novel inputs and should flag them for human review rather than classify them confidently.
• Large-scale image recognition pipelines, since the reported ImageNet results span ResNet-34, ResNet-50, DenseNet-121, and MobileNet-v2.

Industry relevance: Because Catalyst is post-hoc, requires no retraining, preserves ID accuracy, and adds less than 0.0001% (mean) to less than 0.01% (standard deviation) overhead on a ResNet-50 forward pass, it is a candidate for retrofitting onto already-deployed vision models. Its demonstrated compatibility with KNN and with post-hoc pruning methods like ReAct and DICE means it can be layered onto several existing detector families.

Future Directions

• Near-OOD detection. Catalyst's gains on the CIFAR-10 vs. CIFAR-100 task are marginal over SCALE; the authors explicitly name designing a more effective γ for near-OOD settings as an important direction.
• Vision Transformers. The core principle of using pre-pooling statistical cues is described as a general strategy, but adapting it to derive γ from intermediate transformer blocks is stated to require substantial research and engineering; the authors say they are actively exploring this.
• Gradient-based detectors. Extending the framework to gradient-based methods remains future work due to engineering challenges.
• Hyperparameter tuning without a proxy set. The current tuning approach relies on a proxy OOD validation set built with Gaussian noise; a tuning strategy that avoids needing any OOD proxy would be a natural follow-on. The paper also notes that other aggregate functions beyond mean, standard deviation, and max could be explored.

Target Audience

Researchers and practitioners working on reliable or safe deployment of computer vision models — particularly those already using post-hoc OOD detectors such as Energy, ReAct, DICE, ASH, SCALE, or KNN and looking for a cheap add-on. It is also relevant to engineers evaluating whether a small scoring-function change can improve an existing production classifier without retraining. Readers without background in OOD detection metrics (FPR95, AUROC) and CNN feature extraction will find the empirical tables dense, though the core mechanism itself is straightforward.

Note on completeness: the provided paper content is truncated partway through the Related Work section. Per-dataset breakdowns, the clipping-percentile values, the formal characterization in Appendix B, and the comparison against 19 existing OOD detection methods (Appendix F) are referenced but not included here, so those specific numbers are not reported in this summary. One inconsistency appears in the source: the abstract states a 22.25% ImageNet FPR95 reduction on ResNet-50, while the Results section reports a 47.99% reduction for Catalyst(m) versus Energy and a 19.41% improvement for Catalyst(m)+ReAct over previous best; all three figures are reproduced above as written.

Authors’ abstract

Out-of-distribution (OOD) detection is critical for the safe deployment of deep neural networks. State-of-the-art post-hoc methods typically derive OOD scores from the output logits or penultimate feature vector obtained via global average pooling (GAP). We contend that this exclusive reliance on the logit or feature vector discards a rich, complementary signal: the raw channel-wise statistics of the pre-pooling feature map lost in GAP. In this paper, we introduce Catalyst, a post-hoc framework that exploits these under-explored signals. Catalyst computes an input-dependent scaling factor ($γ$) on-the-fly from these raw statistics (e.g., mean, standard deviation, and maximum activation). This $γ$ is then fused with the existing baseline score, multiplicatively modulating it -- an $\textit{elastic scaling}$ -- to push the ID and OOD distributions further apart. We demonstrate Catalyst is a generalizable framework: it seamlessly integrates with logit-based methods (e.g., Energy, ReAct, SCALE) and also provides a significant boost to distance-based detectors like KNN. As a result, Catalyst achieves substantial and consistent performance gains, reducing the average False Positive Rate by 32.87 on CIFAR-10 (ResNet-18), 27.94% on CIFAR-100 (ResNet-18), and 22.25% on ImageNet (ResNet-50). Our results highlight the untapped potential of pre-pooling statistics and demonstrate that Catalyst is complementary to existing OOD detection approaches. Our code is available here: https://github.com/bingabid/Catalyst