On the Problem of Consistent Anomalies in Zero-Shot Industrial Anomaly Detection

AI summary

Overview

• Research area: Computer vision, specifically zero-shot industrial anomaly classification (AC) and anomaly segmentation (AS).
• Technical level: Advanced. The paper combines Vision Transformer feature analysis, graph community detection, and an Extreme Value Theory argument about ordered patch-to-image distances.
• Scope: It introduces CoDeGraph, a graph-based method that detects and filters "consistent anomalies" — recurring, similar defects appearing across many test images — to improve zero-shot AC/AS without any training data.

What This Paper Is About

Zero-shot anomaly detectors score each patch of a test image by comparing it to patches in the other unlabeled test images. This works for one-off defects such as scratches, because no other image contains a matching patch. It fails when a defect is consistent — repeated in many images, such as flipped metal_nut objects — because anomalous patches then find deceptively close matches and receive low anomaly scores. The paper's goal is to identify these consistent anomalies, remove their influence from the comparison set, and thereby improve both classification and segmentation.

Key Contributions

1. Formalizing the consistent-anomaly problem and the "neighbor-burnout" phenomenon. The authors define ϵ-consistent and ϵ-consistent-anomaly patches, then show that normal patches exhibit a stable, gradually increasing distance from their ranked neighbors (power-law decay), whereas consistent-anomaly patches stay close to a limited set of similar images and then spike abruptly once those matches are exhausted. They build an Extreme Value Theory model (Theorem 3.1) predicting that the log similarity growth rate for normal patches follows τ⁽ⁱ⁾(x) ~ Exp(α·i), with expectation 1/(αi) and variance 1/(αi)².
2. The endurance ratio metric. Based on neighbor-burnout, they propose ζ(x, I₍ᵢ₎) = d(x, I₍ᵢ₎) / d(x, I₍ω₎), plus a weighted version ζ′ that multiplies by d(x, I₍ᵢ₎)^(−α) to prioritize links with larger absolute distances, which are more typical of anomalous patches.
3. CoDeGraph, a three-stage framework (identify, isolate, filter). It builds an image-level similarity graph whose nodes are test images and whose edges count suspicious links, so consistent-anomaly images form dense communities; it detects outlier communities with the Leiden algorithm under the Constant Potts Model and Tukey's fences IQR rule; and it selectively removes only patches highly dependent on intra-community matches.
4. State-of-the-art results on consistent-anomaly benchmarks with robustness on conventional ones. The method reports gains of up to 14.9% F1-score and 18.8% AP for segmentation, while leaving performance essentially unchanged on datasets without consistent anomalies.

Main Findings

• Consistent anomalies break existing zero-shot scoring. Because the interval-average score aʙʳ(xₐ) is bounded by H₀ϵ plus the remaining distances, when the number of ϵ-consistent neighbors H₀ is substantial relative to K, consistent-anomaly patches are scored like normal patches.
• Two distinct behavioral signatures. Normal patches and inconsistent anomalies (e.g., in the MVTec AD Cable class) show power-law decay in growth rate, while consistent anomalies show a sudden rise in τ⁽ⁱ⁾(x) after exhausting similar matches. All patches in Capsule, a class with minimal consistent anomalies, follow power-law decay.
• The endurance ratio separates the two populations. The distribution of ζ shows clear separation, while raw absolute distances d(x, I₍ᵢ₎) overlap heavily between normal patches and consistent anomalies, making them hard to distinguish on their own.
• Strong MVTec-CA results with ViT-L/14-336. 98.5% AUROC-cls (+1.3 over the next best zero-shot method), 97.8% F1-cls (+0.8), 99.6% AP-cls (+0.3), 98.1% AUROC-seg (+4.8), 73.8% F1-seg (+14.9), 77.2% AP-seg (+18.8), and 95.4% PRO-seg (+2.6). The paper notes this is comparable to full-shot methods, citing PatchCore-1 at 98.6% AUROC-cls.
• Gains also on the newly introduced benchmarks. On MVTec-SynCA: 96.8% AUROC-cls (+6.9), 97.2% F1-cls (+3.4), 99.0% AP-cls (+2.2), 97.2% AUROC-seg (+6.5), 63.2% F1-seg (+12.4), 63.3% AP-seg (+15.0), 91.1% PRO-seg (+8.2). On ConsistAD: 91.0% AUROC-cls (+2.1), 87.9% F1-cls (+3.1), 90.3% AP-cls (+1.4), 86.9% AUROC-seg (+5.2), 55.9% F1-seg (+11.3), 57.5% AP-seg (+13.9), 82.5% PRO-seg (+3.6).
• No loss on inconsistent-anomaly data. On MVTec-IA, CoDeGraph records 98.3% AUROC-cls, 97.3% F1-cls, 99.1% AP-cls, 98.2% AUROC-seg, 65.1% F1-seg, 65.8% AP-seg, 94.4% PRO-seg; on Visa, 91.6% AUROC-cls, 89.0% F1-cls, 92.2% AP-cls, 98.7% AUROC-seg, 48.3% F1-seg, 45.4% AP-seg, 91.4% PRO-seg — matching MuSc closely.
• Filtering is targeted, not wholesale. On MVTec-CA the method excluded 6.9% of patches from the base set while capturing 73.9% of consistent-anomaly patches (distribution 39.3% normal / 31.5% consistent / 29.2% inconsistent). On ConsistAD it excluded 4.8% of patches and captured 55.2% of consistent anomalies. On inconsistent datasets the exclusion rate is 0.3% for MVTec-IA and 0.05% for Visa, with a capture rate of 0.9% and 0.0% respectively.
• Filtering is the safeguard. Without patch filtering, Top-3 community selection removed 12.5% of normal patches on MVTec-CA (versus 2.8% with filtering) and 3.2% on MVTec-IA (versus 0.2% with filtering), reducing MVTec-IA pixel-wise F1 by 2.9% and image-wise AUROC by 2.0%.
• Hyperparameter insensitivity. Results stayed nearly unchanged for Tukey's fences settings k_IQR = 1.5 and 3 instead of the default 4.5.
• Backbone generality. With a DINOv2-L-14 backbone, segmentation improved further to 89.9% pixel-wise AUROC, 69.1% pixel-wise F1, and 71.9% pixel-wise AP, described as +6.5% F1 and +9.2% AP over prior state of the art.
• Abstract headline numbers. The abstract states 98.3% AUROC for AC on MVTec AD with ViT-L-14-336 and AS performance of 66.8% (+4.2%) F1 and 68.1% (+5.4%) AP over state-of-the-art zero-shot methods.

Methodology in Plain English

The method starts from an existing zero-shot scorer (MuSc) that ranks, for each patch, how far it is from the nearest patch in every other test image. The authors observe that a patch belonging to a repeated defect will have a few very close matches and then a sudden jump, while a normal patch's distances grow smoothly. They turn that observation into a single ratio: compare the distance to the i-th nearest image with the distance to a much later-ranked image. A small ratio is a red flag — the patch links suspiciously strongly to one particular other image.

They then build a graph whose nodes are test images. An edge is drawn between two images whenever a flagged ("suspicious") link connects a patch across them, and the edge weight counts how many such links exist. Because repeated defects link the same small set of images to each other many times, consistent-anomaly images clump into dense communities that stand out from the sparse rest of the graph. A coverage rule gradually admits more links until 95% of images have at least one connection, which keeps the graph from being swamped by ordinary links. The weighted endurance ratio (with α = 0.2) further biases the flagged links toward anomalous patches rather than normal ones.

Communities are found with the Leiden algorithm using the Constant Potts Model, whose resolution parameter is set to the 25th quantile of edge weights. Communities whose density exceeds Tukey's fences threshold with k_IQR = 4.5 are treated as consistent-anomaly communities. Rather than deleting those images outright, the method removes only the individual patches inside them whose score would rise sharply if the community were removed — the threshold is the 99th percentile of ratios computed from patches outside the community. The remaining patches form the new base set, and final anomaly scores are recomputed with the original mutual scoring mechanism using receptive fields r ∈ {1, 3, 5}.

Implementation uses ViT-L/14-336 (24 layers, four stages of six layers) with patch tokens from layers 6, 12, 18 and 24, images resized to 518×518, an interval-average of the lowest 10% of distances instead of the 30% used previously, ω = 0.3·N, coverage target τ = 0.95, and a linearly projected final-layer class token optimized with RsCIN for classification. Reported experiments used MVTec AD (15 classes), split into MVTec-CA (cable, metal_nut, pill) and MVTec-IA (the remaining 12 classes), plus Visa, alongside the new MVTec-SynCA and ConsistAD benchmarks. Consistent anomalies are defined for evaluation as anomalous patches whose scores fall below the 80th percentile of normal patch scores; the paper states this ground-truth threshold is used only for post-hoc analysis and evaluation, not for the method itself.

Why This Matters

Impact on research. Zero-shot anomaly detection has largely been framed as a patch-level or prompt-level matching problem. This paper argues that image-level structure in the test set itself carries information that patch-level comparisons discard, and backs that argument with an Extreme Value Theory model rather than only empirical observation. It also introduces two benchmarks (MVTec-SynCA and ConsistAD) for a failure mode that prior evaluation protocols did not isolate, which gives the community a way to measure progress on the problem.

Real-world applications.

• Automated visual inspection on manufacturing lines, where a batch of parts may share the same defect (for example, all metal_nut objects flipped, or components missing from every cable), and per-patch similarity scoring would quietly under-report it.
• Pharmaceutical pill inspection, where contaminated pill images form repeated defect patterns within a production run.
• Logical and structural defect detection in assembled products, such as the breakfast_box items in MVTec LOCO mentioned for high normal-pattern variability.
• Deployment scenarios where no labeled training data exists and only a stream of unlabeled test images is available, which is the zero-shot setting the method targets.

Industry relevance. The method requires no training data and, according to the paper, uses fixed hyperparameters across all datasets with no per-dataset tuning. It also leaves performance essentially unchanged on datasets without consistent anomalies (e.g., 98.3% AUROC-cls on MVTec-IA and 91.6% on Visa, matching MuSc), which matters for a general-purpose inspection system that cannot know in advance whether a given product batch will exhibit repeated defects.

Future Directions

• Relaxing the modeling assumptions. The Extreme Value Theory derivation assumes a power-law tail in patch similarity distributions and treats distances as i.i.d. The paper acknowledges spatial correlations exist and justifies the approximation by the large number of patches per image (e.g., 1369), leaving a more faithful model of correlated distances as an open question.
• Extending beyond industrial imagery. The neighbor-burnout intuition is derived from industrial product images where objects are semantically identical. Whether the endurance ratio remains discriminative in domains with high intra-class variation is not established in the reported experiments.
• Characterizing when filtering can hurt. Results are reported on MVTec AD, Visa, MVTec-SynCA and ConsistAD, with exclusions of 0.3% and 0.05% of patches on the two standard benchmarks. The conditions under which graph communities could be spuriously formed on novel data are not reported.
• Further backbone and architecture studies. The paper reports DINOv2-L-14 improving segmentation to 89.9% pixel-wise AUROC, 69.1% pixel-wise F1 and 71.9% pixel-wise AP, and places additional results on different ViT structures in its appendix, suggesting the choice of feature extractor is an active variable.

Target Audience

Researchers and engineers working on zero-shot anomaly detection, industrial visual inspection, or unsupervised defect segmentation, particularly those already familiar with representation-based methods such as MuSc and with Vision Transformer patch features. The paper is also relevant to applied machine-learning practitioners who need a training-data-free anomaly detector and want to understand a failure mode (repeated defects) that benchmark averages can hide. Readers need some comfort with graph community detection and order statistics to follow the theoretical section, though the core intuition — normal patches decay smoothly, repeated-defect patches burn out — is accessible without it.

Authors’ abstract

Zero-shot image anomaly classification (AC) and segmentation (AS) are vital for industrial quality control, detecting defects without prior training data. Existing representation-based methods compare patch features with nearest neighbors in unlabeled test images but struggle with consistent anomalies -- similar defects recurring across multiple images -- resulting in poor AC/AS performance. We introduce Consistent-Anomaly Detection Graph (CoDeGraph), a novel algorithm that identifies and filters consistent anomalies from similarity computations. Our key insight is that normal patches in industrial images show stable, gradually increasing similarity to other test images, while consistent-anomaly patches exhibit abrupt similarity spikes after exhausting a limited set of similar matches, a phenomenon we term ``neighbor-burnout.'' CoDeGraph constructs an image-level graph, with images as nodes and edges connecting those with shared consistent-anomaly patterns, using community detection to filter these anomalies. We provide a theoretical foundation using Extreme Value Theory to explain the effectiveness of our approach. Experiments on MVTec AD with the ViT-L-14-336 backbone achieve 98.3% AUROC for AC and AS performance of 66.8% (+4.2%) F1 and 68.1% (+5.4%) AP over state-of-the-art zero-shot methods. Using the DINOv2 backbone further improves segmentation, yielding 69.1% (+6.5%) F1 and 71.9% (+9.2%) AP, demonstrating robustness across architectures.