Anomaly Detection and Industrial Inspection

How much data you have changes the problem more than the model family name. Supervised defect detection learns labelled examples of known defect types, and misses new failure modes. One-class or feature-distance detection models normal features and flags departures: abundant normal data, at the price of sensitivity to nuisance change. Synthetic-defect training fabricates plausible perturbations to teach localisation. It expands rare coverage and controls defect masks, and it carries simulation mismatch risk. A hybrid review system combines anomaly score, rules, known-defect heads and human judgement, which separates severity from novelty but requires policy design.

The synthetic column has a named method behind it, with a number and a candid warning. DRÆM — Zavrtanik and colleagues, 2021 — trains a defect detector on simulated anomalies only: binarised Perlin-noise masks blended with textures from the unrelated Describable Textures Dataset. Across the fifteen MVTec AD classes it reaches a mean image-level AUROC of 98.0. That is 2.5 percentage points above the previous best, and 13.4 percentage points better in pixel-level average precision. What makes it worth teaching is that its authors never claim the fabricated defects resemble real ones. The paper says the opposite: “DRÆM does not require simulations to realistically reflect the real anomaly appearance in the target domain, but rather to generate just-out-of-distribution appearances, which allow learning the appropriate distance function to recognize the anomaly by its deviation from normality.” The simulator is teaching a distance, not a defect.

The risk in that column is measurable too, not just rhetorical. Engineers at Valeo and Intel re-ran DRÆM alongside ten other models over nine datasets in 2025. The generation-reliant discriminative methods were the ones that failed on the harder real-production sets. Choose the synthetic route knowing what it optimises: an out-of-distribution boundary, validated against real defects you did not draw.

A robust inspection system models several sources of change stacked on top of each other: acquisition variation (illumination, focus, exposure, vibration, lens contamination), product variation (colour, lot, supplier, material, orientation, acceptable texture), process state (machine, tool wear, shift, line speed, maintenance event), defect evidence (scratch, crack, missing part, contamination, deformation), and the decision policy on top — accept, rework, stop line, sample review, or investigate.

The first layer of that stack is the one that has now been measured, and it is expensive. MVTec AD 2, published by MVTec in 2025, is a set of eight industrial inspection scenarios built from “more than 8,000 high-resolution images”. Its test splits deliberately contain lighting conditions that are not necessarily present in training. The abstract states the intent plainly: “Additionally, our dataset provides test scenarios with lighting condition changes to assess the robustness of methods under real-world distribution shifts.”

Then comes the number to carry away. State-of-the-art anomaly detection methods — the same families that compete within one percentage point of each other on MVTec AD — score below 60 per cent average AU-PRO on MVTec AD 2. The VAND 3.0 challenge at CVPR 2025, run by Voxel51 with Intel and MVTec, made this its first category: distribution shifts caused by lighting, camera angle and noise.

Acquisition variation is therefore not a preliminary to the modelling work. Moving a lamp costs more performance than any architecture choice in this literature has ever gained.

Before changing the neural architecture, inspect these sources. Each one changes the image without changing the product. That is the same kind of shift that separates saturated scores on MVTec AD from below 60 per cent average AU-PRO on MVTec AD 2.

A low threshold catches subtle anomalies and increases review or line stoppage. A high threshold reduces false alarms and lets weak defects through. The threshold, not the architecture, is where the model meets the plant's budget.

Benchmark scores cannot choose it for you, and that is now a published finding. Valeo and Intel ran eleven state-of-the-art anomaly detection models across nine datasets and seven metrics in 2025, and concluded: “Our extensive experiments across nine datasets, eleven SOTA models, and seven metrics reveal that current evaluation practices may not reliably predict deployment success, as models with 99.9% image-level AUROC on MVTecAD show significant degradation on real-world data.”

Their remedy is to report an operating point rather than an area under a curve. Presorted Good at 2 per cent — PG2 — is the share of good parts correctly cleared while 2 per cent of bad parts are missed, with its counterpart PB2 beside it. That is a sentence a production manager can price: this many parts skip human review, at this rate of escape. An image-level AUROC of 99.9 per cent is not.

A separate group arrived at the same place from the other side. Real-IAD is a 150,000-image, 30-object multi-view dataset, presented at CVPR 2024. It was built precisely because saturation above 99 per cent AUROC on mainstream datasets such as MVTec had left a significant gap between public datasets and actual application scenarios.

Use severity-aware policies, product variants, operating modes and review capacity. Calibrate thresholds on chronological or lot-based splits that resemble deployment.

Random image splits can leak nearly identical production conditions, so the plan runs in five moves. Split by production unit rather than by adjacent frames. Test known defects by severity and size, for classification and localisation. Test unknown deviations by holding out defect families or introducing controlled process changes. Audit false alarms for acquisition, acceptable variation and recurring nuisance clusters. And measure operations: review load, escaped defects, rework, downtime and threshold changes.

Steps two and five are not house style. For automated inspection of sterile products they are binding, in EU GMP Annex 1 paragraph 8.32: “Where automated methods of inspection are used, the process should be validated to detect known defects (which may impact product quality or safety) and be equal to, or better than, manual inspection methods. The performance of the equipment should be challenged using representative defects prior to start up and at regular intervals throughout the batch.” Note the shape of that sentence. Validation is periodic and adversarial — representative defects presented again during the batch, not an acceptance test signed once. And the bar is manual inspection, not a leaderboard. Paragraph 8.31 governs the acquisition side of manual inspection, requiring “suitable and controlled conditions of illumination and background” at qualified inspection rates.

Step two cannot end in a yes or a no, and FDA's draft guidance explains why in a footnote: “Visual detection of a particulate is a probabilistic process that depends on, among other things, the product and the size and shape of the particulate”. The guidance therefore asks for operating characteristic curves for each defect classification, alongside accept and reject criteria, an AQL and a rejectable quality limit. One curve per defect class, showing detection as a function of how bad the defect is. That is what step five's cost accounting looks like once it reaches a plant, and it is the same object as the operating-point metrics the Valeo and Intel team asked researchers to publish.

A line produces many acceptable texture variants and very few examples of leakage-causing microcracks. Design the system the way the evidence above forces you to: a taxonomy before a threshold, an operating point before an AUROC, and a reference set whose contents you can account for.