SupLID: Geometrical Guidance for Out-of-Distribution Detection in Semantic Segmentation

AI summary

Overview

Research area: Out-of-distribution (OOD) detection for semantic segmentation, combining computer vision reliability with non-parametric geometric analysis of high-dimensional feature spaces.

Technical level: Intermediate. The paper assumes familiarity with semantic segmentation, classifier confidence scores (energy, entropy, MaxLogit), and the concept of intrinsic dimensionality.

Scope: The paper introduces SupLID, a post-hoc scoring framework that multiplies a standard classifier-based OOD score by a Local Intrinsic Dimensionality (LID) term computed over a compact geometric coreset of superpixel embeddings, and evaluates it on five OOD benchmarks across several segmentation backbones.

What This Paper Is About

Semantic segmentation models in safety-critical settings such as autonomous driving must flag individual pixels that do not belong to any trained class, not just label whole images as anomalous. Existing pixel-wise OOD detectors mostly reuse image-level confidence scores (energy, entropy, MaxLogit, KL, MSP), which are prone to overconfidence on far-OOD objects and produce noisy, fragmented anomaly maps. SupLID's goal is to supply a complementary geometric signal — derived from the dimensionality of the data's local semantic structure — that corrects these failures without retraining the segmentation network.

Key Contributions

1. A geometric guidance mechanism for OOD scores. SupLID augments any classifier-based pixel-wise OOD score with a term derived from Local Intrinsic Dimensionality (LID), computed on a tampered reference subspace. The paper reports this as the first exploration of geometric structure to enhance classifier-based OOD detection in semantic segmentation, operating in a fully post-hoc manner.

2. A novel LID-based geometrical coreset. For each in-distribution (ID) class, SupLID selects a subset of m superpixel embeddings with the lowest LID, weighted by their own LID values, to form a compact reference subspace. This reduces the number of neighborhood queries required for LID estimation from N × |Z| to K × m, which the authors state is on the order of 10², making scalable pixel-wise inference feasible.

3. Superpixel-based aggregation. Using the classical SLIC algorithm, scores are computed on spatially coherent class-agnostic regions rather than individual pixels, reducing computational complexity by a factor of H × W / N and improving spatial smoothness and boundary consistency of anomaly masks.

4. Broad empirical evaluation. SupLID is tested both as a standalone post-hoc score and as an add-on to retraining-based methods (PEBAL, MetaOOD, BE-OE, RPL) across RoadAnomaly, SMIYC, and Fishyscapes benchmarks, on both public validation sets and the hidden SMIYC online test sets.

Main Findings

• SupLID improves all five post-hoc baselines on average metrics. On DeepLabv3+ with a WideResNet-38 backbone, SupLID achieves average FPR 33.32, AUP 45.40, and AUR 91.55 across the five benchmarks, compared with, for example, energy (FPR 40.34, AUP 31.44, AUR 88.43), MaxLogit (40.54 / 32.03 / 88.26), and NNGuide (38.74 / 24.37 / 88.08). Lower FPR is better.

• Gains are largest on SMIYC. On SMIYC-Anomaly, SupLID reaches FPR 44.55, AUP 59.46, AUR 87.94; on SMIYC-Obstacle, FPR 2.27, AUP 68.96, AUR 99.36. The authors attribute this to SupLID's robustness to context shift, since SMIYC features contexts outside the training ID set.

• Non-parametric baselines underperform. Mahalanobis (average FPR 63.75, AUP 16.25, AUR 70.30) and kNN (54.50 / 13.62 / 77.56) trail the classifier-based and SupLID scores by wide margins.

• SupLID boosts retraining-based SOTA models. Applied on top of existing methods using their default confidence scores: PEBAL improves on SMIYC-Anomaly from FPR 36.49, AUP 53.81, AUR 89.07 to 28.14 / 69.13 / 92.51; BE-OE improves from 58.95 / 45.28 / 83.68 to 39.19 / 59.14 / 89.03; MetaOOD improves from 17.46 / 80.76 / 95.91 to 15.45 / 82.13 / 96.43; RPL improves modestly from 7.18 / 88.55 / 98.06 to 6.74 / 89.77 / 98.24.

• Improvement on RPL is the smallest. The paper explains that RPL's performance is already saturated due to its context-aware contrastive learning, while noting that RPL requires complex retraining whereas SupLID is plug-and-play.

• Gains hold on the hidden SMIYC online test sets. Energy goes from FPR 72.96, AUP 33.66, F1 6.56 to 35.48 / 55.84 / 16.28 on SMIYC-Anomaly, and from 15.54 / 24.81 / 11.80 to 11.72 / 59.77 / 33.04 on SMIYC-Obstacle. PEBAL improves from 40.73 / 49.33 / 14.58 to 27.81 / 66.72 / 24.48 (Anomaly) and from 12.69 / 4.95 / 5.72 to 11.47 / 56.78 / 39.96 (Obstacle). MetaOOD improves from 14.96 / 85.46 / 34.56 to 10.26 / 85.65 (remaining values truncated in the provided content).

• ID segmentation quality is preserved. The paper states the method achieves the same mean Intersection over Union (mIoU) as the original model, since no retraining or architecture change is involved.

• Configured hyperparameters. The paper selects k = 400 neighborhood samples and m = 400 coreset entries per class from a candidate set of {10, 20, 50, 100, 200, 300, 400, 500, 600, 800, 1000, 2000}, and sets N by dividing total pixels by a factor of 200, chosen from {10, 50, 100, 150, 200, 250, 300, 350, 400, 500}. LID is computed on a penultimate feature layer of dimension 304.

Methodology in Plain English

The starting point is that a classifier's own confidence — such as the energy score — is a good signal only near decision boundaries. Far-OOD objects can look confident, and difficult ID pixels can look uncertain. SupLID adds a second, independent opinion based on geometry.

First, the image is broken into superpixels with SLIC, grouping pixels that are spatially and visually coherent. Averaging the classifier score within each superpixel reduces noise and computation, and the segmentation is class-agnostic so unseen objects are treated like anything else.

Second, the authors build a reference set from training data. Every training superpixel is embedded by the network's encoder, and for each ID class the m embeddings with the lowest LID are kept. Low LID means the point sits in a compact, low-dimensional neighborhood — the "core" of the class. These selected points are then deliberately modified: features with high LID (least salient) are amplified and low-LID features suppressed, which turns the tidy low-dimensional manifold into a noisy, high-dimensional one. The intuition, drawn from prior LID work on adversarial examples and backdoored data, is that ID samples will appear anomalously high-dimensional in this tampered subspace while far-OOD samples will not.

Third, for each test superpixel embedding, the LID is estimated by the maximum-likelihood estimator relative to the k nearest weighted coreset points. This LID value is used as the geometric factor D. The final score is simply the classifier score S multiplied by D, and the superpixel value is broadcast back to all pixels in that region.

Why This Matters

Research impact: The paper reframes pixel-wise OOD detection as a problem of combining classifier confidence with an explicit geometric characterization of the semantic space, showing that a non-parametric dimensional measure can complement — and correct — parametric confidence scores. It also demonstrates that LID, previously applied to adversarial subspace detection and noisy-label mitigation at the instance level, can be made tractable at pixel level through coreset and superpixel reduction.

Real-world applications:

• Autonomous driving: flagging unexpected objects on the road (the RoadAnomaly and SMIYC-Obstacle benchmarks are drawn from this setting) so a vehicle can slow or take evasive action.
• Robotics and industrial inspection: detecting foreign objects or defects on a production line where the model was trained only on known object categories.
• Medical imaging: highlighting pathology or artifacts that fall outside the annotated classes the segmentation model learned.
• Deployment-time safety monitoring: acting as a post-hoc guard on already-deployed segmentation models, since no retraining or architectural change is needed.

Industry relevance: Because SupLID is a post-hoc scoring method, it can be layered onto an existing trained segmentation network at deployment without the cost, engineering effort, or risk of a retraining pipeline. The paper contrasts this with reconstruction-based hybrid approaches (SynBoost, DenseHybrid), which need an additional network that is hard to train and inefficient for real-time inference, and with retraining-based methods (PEBAL, MetaOOD, RPL) that require modifying the training procedure.

Future Directions

1. Reducing the remaining gap on saturated models. RPL gained only marginally from SupLID; understanding when the geometric signal adds nothing, and whether it can be reformulated to help context-aware models, remains open.

2. Choosing hyperparameters without validation OOD data. The paper tunes k, m, and N using a validation strategy, but these depend on the ID dataset and backbone; a principled, automatic selection rule would improve practicality.

3. Extending beyond the tested OOD benchmarks. The evaluation covers urban driving datasets (Cityscapes as ID; SMIYC, Fishyscapes, RoadAnomaly as OOD) with one primary backbone family; generality to other domains such as medical or aerial imagery is not established in the provided content.

4. Improving on LID's local nature. The paper itself notes in Figure 1 that LID may fail to capture full anomalous objects due to its localized character; how much of this limitation survives superpixel aggregation, and whether multi-scale variants would help, is left for further work.

Target Audience

Researchers and practitioners working on OOD detection, anomaly segmentation, or trustworthy computer vision — particularly those interested in non-parametric and geometric approaches rather than retraining. It is also relevant to engineers who need to add safety monitoring to an already-deployed segmentation model, since the method requires no retraining. Readers without background in semantic segmentation metrics or intrinsic dimensionality will need to consult the cited prior work (Houle et al. on LID, Tian et al. on PEBAL) to follow the technical details.

Authors’ abstract

Out-of-Distribution (OOD) detection in semantic segmentation aims to localize anomalous regions at the pixel level, advancing beyond traditional image-level OOD techniques to better suit real-world applications such as autonomous driving. Recent literature has successfully explored the adaptation of commonly used image-level OOD methods--primarily based on classifier-derived confidence scores (e.g., energy or entropy)--for this pixel-precise task. However, these methods inherit a set of limitations, including vulnerability to overconfidence. In this work, we introduce SupLID, a novel framework that effectively guides classifier-derived OOD scores by exploiting the geometrical structure of the underlying semantic space, particularly using Linear Intrinsic Dimensionality (LID). While LID effectively characterizes the local structure of high-dimensional data by analyzing distance distributions, its direct application at the pixel level remains challenging. To overcome this, SupLID constructs a geometrical coreset that captures the intrinsic structure of the in-distribution (ID) subspace. It then computes OOD scores at the superpixel level, enabling both efficient real-time inference and improved spatial smoothness. We demonstrate that geometrical cues derived from SupLID serve as a complementary signal to traditional classifier confidence, enhancing the model's ability to detect diverse OOD scenarios. Designed as a post-hoc scoring method, SupLID can be seamlessly integrated with any semantic segmentation classifier at deployment time. Our results demonstrate that SupLID significantly enhances existing classifier-based OOD scores, achieving state-of-the-art performance across key evaluation metrics, including AUR, FPR, and AUP. Code is available at https://github.com/hdnugit/SupLID.