TriScore: Post-Hoc Out-of-Distribution Detection with Energy, Boundary Probes, and Transform Consistency

AI summary

Overview

Research area: Computer Vision — post-hoc out-of-distribution (OOD) detection and robustness of frozen image classifiers.

Technical level: Advanced. The paper assumes familiarity with logits, softmax, energy scores, gradients, and standard OOD metrics such as AUROC and FPR@95.

Scope: The paper introduces TriScore, a post-hoc OOD detector that fuses three ID-calibrated signals from a single frozen backbone and evaluates it on four ImageNet-scale architectures under both clean OOD benchmarks and a non-adaptive adversarial-shift protocol.

What This Paper Is About

Most post-hoc OOD detectors rely on one signal from a frozen classifier — confidence, energy, features, or gradients. If a shift happens to preserve that particular statistic while changing the model's decision behavior, the detector has no backup and can degrade toward random guessing. The paper asks whether combining several complementary, ID-calibrated signals from the same frozen backbone can raise the worst-case robustness floor without retraining, OOD labels, an auxiliary model, or a stored feature bank.

Key Contributions

1. TriScore, a three-branch post-hoc detector. It combines a class-centered energy score, a Boundary-Probe fragility score (a top-k margin divided by the gradient of that margin at a late feature representation), and a Transform-Consistency score (Jensen-Shannon divergence between the base prediction and two mild deterministic views).

2. An ID-residual gated fusion rule. Each branch is z-scored on ID validation data, and a per-sample softmax gate upweights whichever branch most exceeds its ID-tail quantile. Calibration uses only ID data, requires no OOD or adversarial examples, and stores no feature bank.

3. A large-scale adversarial-shift evaluation. Using the OpenOOD framework across four ImageNet-scale backbones (ResNet-50, RegNet, ViT-B/16, Swin-T) and 15 post-hoc baselines, TriScore achieves the best mean AUROC on all four backbones under a 10-attack, non-adaptive protocol.

4. A controlled ablation study of complementarity. The paper shows the gains come from combining cues that fail for different reasons rather than from optimizing one detector signal, and it examines whether fusing other existing detectors can reach the same floor.

Main Findings

• Best mean AUROC under adversarial shift on all four backbones. On ResNet-50 TriScore reaches 62.11 AUROC; on RegNet 72.04 (versus 71.09 for PRO and 70.29 for EBO); on ViT-B/16 63.11 (versus 61.99 for GEN); on Swin-T 65.91 (versus 65.43 for GEN and 65.24 for DICE).

• Best mean FPR@95 on three of four backbones. TriScore reports 71.26 (RegNet), 84.10 (ViT-B/16), and 80.01 (Swin-T). On ResNet-50 it scores 88.64, with PRO, MSP, and CADRef lower in that column.

• Broad rather than attack-specific gains. Across 40 attack × backbone cells, TriScore has the best AUROC in 27 and the best FPR@95 in 19, compared with 16 for PRO. Per-attack tables show no single attack drives the aggregate result.

• Lower cross-attack variance than the closest high-AUROC competitors. On Swin-T the per-attack AUROC standard deviation is 4.5 for TriScore versus 5.5 for GEN and 6.9 for DICE; on ViT-B/16 it is 5.5 versus 6.6 for GEN and 5.8 for DICE. Some lower-mean baselines have smaller variance, so the claim is relative to strong competitors.

• A higher robustness floor across architectures. TriScore's worst-backbone adversarial mean AUROC is 62.1 (ResNet-50), above the best baseline floor of 59.4 (PRO). Several baselines fall below 50 on at least one backbone (GradNorm 47.0, ODIN 47.8).

• Competitive but not dominant on clean OOD. TriScore's four-backbone clean mean AUROC is 82.67, with seven baselines above it (CADRef 87.12, GEN 86.23, ViM 85.44, PRO 83.84, KNN 83.48, NCI 83.43, MLS 82.76). The authors attribute this to TriScore not exploiting richer ID-training feature geometry.

• Feature-based baselines lose more under shift and cost more. CADRef, ViM, and KNN lose 26–28 AUROC from clean to adversarial averaged over the four backbones, versus 16.9 for TriScore, and their adversarial AUROC (55–63) falls below TriScore's (62–72) on every backbone. KNN additionally retains and searches a 1.28M-vector feature bank.

• Fewer architecture-specific collapses on clean OOD. TriScore stays at or above 80 AUROC and below 66 FPR@95 on all four clean backbones, while ASH (52.38 on ViT-B/16), GradNorm (56.86 on RegNet), DICE (54.46 on Swin-T), and SCALE (66.97 on ViT-B/16) each collapse on at least one.

• A cheaper E+BP variant. The two-branch subset has slightly higher clean AUROC than gated TriScore on ResNet-50 and RegNet, while the gated detector has higher AUROC on ViT-B/16 and Swin-T and lower FPR@95 on all four.

Methodology in Plain English

The researchers start from a classifier that is already trained and never modify it. They extract three different kinds of evidence from it.

The first is energy: instead of looking only at the top predicted class, it summarizes the whole logit vector. Before computing it, they subtract the average logit vector measured on in-distribution validation images, so that classes which always score high do not distort the signal.

The second, Boundary-Probe, asks how fragile the current decision is. It measures how far the top logit sits above the average of the next few, then asks how quickly that gap would shrink if the internal late-layer features were nudged. A large gap that shifts quickly, or a small gap, counts as fragile.

The third, Transform-Consistency, asks whether the prediction stays stable when the image is flipped or center-cropped to 90% and resized. Rather than comparing only the top label or the entropy, it compares the full probability distributions using Jensen-Shannon divergence.

Each of the three numbers is standardized using statistics from an in-distribution validation split. A gate then looks at how far each standardized score exceeds a fixed in-distribution quantile and gives more weight to whichever branch is most unusual for that particular image. All calibration constants, temperatures, and thresholds come from ID data only, using at most 5,000 images; the evaluation never lets an attacker see or optimize the detector.

Evaluation uses OpenOOD with ImageNet-1K as in-distribution, SSB-Hard and NINCO as near-OOD, and iNaturalist, Textures, and OpenImage-O as far-OOD. The adversarial protocol applies ten attacks (AutoAttack, APGD-CE, C&W, DeepFool, FGSM, masked PGD, PGD, Square, BIM, and PGD-BPDA), with ℓ∞ attacks at ε = 4/255, and scores the perturbed ImageNet images as shifted inputs.

Why This Matters

Impact on research. The paper reframes post-hoc OOD detection as a redundancy problem rather than a single-score optimization problem, and it argues that mean clean AUROC hides architecture-specific failures that matter in deployment. It also supplies a non-adaptive adversarial-shift protocol that exposes which detectors collapse and on which backbones.

Real-world applications:

• Safety monitoring in deployed vision systems (automotive perception, robotics) where a classifier is already frozen in production and an OOD check must be bolted on afterward.

• Model auditing and selection when teams compare candidate backbones and need a detector whose performance does not silently degrade on one architecture.

• Open-set recognition in data pipelines, such as flagging inputs that should be routed to human review before entering a labeled dataset.

• Resource-constrained deployment, since the method needs no retraining, no auxiliary model, no OOD data, and no stored feature bank — unlike KNN-style detectors that keep millions of feature vectors.

Industry relevance. The ID-only calibration requirement (at most 5,000 images) and the absence of OOD data or retraining make the method compatible with locked-down production models and with privacy or labeling constraints. Its main trade-off is explicit: it gives up a few points of clean OOD accuracy for a stronger robustness floor.

Future Directions

• Closing the clean-OOD gap. TriScore trails CADRef, GEN, ViM, PRO, KNN, NCI, and MLS on four-backbone clean mean AUROC (82.67) because it does not use the richer ID-training feature geometry those methods exploit; combining that geometry with the gate is an open question.

• Adaptive attacks. The reported adversarial protocol is explicitly non-adaptive; the paper points to an appendix study of adaptive attacks, but the headline claims rest on an attacker that does not optimize the detector or the gate.

• Stronger feature-coordinate invariance. The Boundary-Probe score is shown to depend on the chosen feature coordinates, with the dependence cancelling only under the ID z-scoring after a global rescaling; more coordinate-stable fragility measures remain open.

• Generalizing beyond the gate's fixed quantiles and softness. The gate uses fixed ID-only defaults (p_E = 0.90, p_BP = 0.70, p_C = 0.70, s = 0.6); whether these transfer to other data regimes, and whether other existing detectors can be fused to reach the same floor, is raised by the paper's own follow-up analyses.

Target Audience

Researchers and engineers working on out-of-distribution detection, open-set recognition, and robustness of vision models — particularly those deploying frozen classifiers who cannot retrain or access outlier data. It is also relevant to practitioners choosing between post-hoc detectors across architectures, and to reviewers interested in how evaluation protocol design (clean versus adversarial shift) changes which method looks best. Readers need a working knowledge of logits, softmax, gradients, and OOD metrics such as AUROC and FPR@95.

Authors’ abstract

Post-hoc out-of-distribution detection usually relies on a single signal extracted from a frozen classifier, such as confidence, energy, features, or gradients. This dependence becomes fragile under classifier-level adversarial shift, where the shift can preserve the detector's chosen cue while changing the model's decision behavior. We investigate whether this failure mode can be mitigated by combining complementary detection criteria instead of selecting a single score. We introduce TriScore, a post-hoc OOD detector that combines three ID-calibrated signals from the same frozen backbone: an energy prior, a boundary-fragility score, and a transform-consistency score. These signals are standardized on ID validation data and fused by an input-dependent ID-residual gate, requiring no OOD labels, auxiliary model, training, or stored feature bank. Across four ImageNet-scale backbones, TriScore achieves the best mean AUROC among 15 post-hoc baselines under a non-adaptive adversarial-shift protocol, while remaining competitive on standard OOD benchmarks. Ablations show that the gains come from combining complementary cues rather than optimizing a single detector signal. These results support multi-criterion post-hoc detection as a practical route toward more robust OOD detection under classifier-level shift. The source code is publicly available at github.com/gursoyege/TriScore.