Detecting Generated Images by Fitting Natural Image Distributions

AI summary

Overview

• Research area: Computer vision — detecting AI-generated images (often framed as a binary classification problem), with connections to self-supervised learning, manifold geometry, and normalizing flows.
• Technical level: Advanced. The detection procedure itself is simple and training-free, but the design is derived from a manifold/orthogonality argument involving tangent spaces, Jacobians, and change-of-variables formulas for flows.
• Scope (one sentence): The paper proposes ConV (consistency verification) and its flow-based extension F-ConV, which detect generated images by testing whether an image is consistent with a distribution fitted only on natural images, rather than by training a classifier on generated ones.

What This Paper Is About

Most detectors are binary classifiers trained on a collection of natural and generated images, so their accuracy depends on how many generated images you can gather and how well they represent the generator you will face at test time. That is a problem because new generators keep appearing, and some (for example Sora) are not accessible for data collection or retraining. This paper asks whether a detector can instead be built to rely only on the distribution of natural images, and shows how to do it by exploiting the geometric gap between the natural-image manifold and generated images.

Key Contributions

1. Framing the problem as a generalization failure: The authors argue that existing detectors' performance hinges on the diversity of generated data, and that it is hard to know whether a detector trained on images from some diffusion models will transfer to images from other models. They motivate detectors whose ability depends only on fitting the natural data distribution.
2. ConV, a consistency-verification framework: They introduce two functions designed to output the same value on natural images but different values on generated ones. The design follows an explicit orthogonality principle (Eq. 6): the functions' gradients must lie in two mutually orthogonal subspaces — the tangent space of the natural manifold and its orthogonal complement.
3. A training-free detector: Because the two functions can be instantiated from an existing self-supervised model, detection reduces to checking whether a transformation along the data manifold substantially changes the model's loss. The paper implements this with DINOv2 and standard training-time augmentations, with no detector training required.
4. F-ConV, manifold extrusion with normalizing flows: To handle the shrinking gap between natural images and those from newer generators, they use a normalizing flow (RealNVP) to map the natural-image manifold to a Gaussian and push generated images away from it, using a shaping loss plus a consistency loss.

Main Findings

• Training-free results are competitive with trained detectors: On the ImageNet benchmark across ADM, ADMG, LDM, DiT, BigGAN, GigaGAN, StyleGAN XL, RQ-Transformer, and Mask GIT, ConV (no training) reaches an average AUROC of 87.13 and AP of 85.15, versus 93.68/93.11 for FatFormer, 90.36/89.92 for DRCT, 86.00/80.84 for NPR, and 85.35/84.25 for Ojha among training-based methods, and 57.87/57.85 for the training-free baseline AEROBLADA.
• Flow-based extrusion gives the best average: F-ConV reaches 93.77 AUROC and 93.38 AP on the same ImageNet evaluation, slightly above FatFormer's 93.68/93.11.
• Strong results on inaccessible video generators: On 1,000 frames gathered from Sora videos, ConV scores 87.74 AUROC and 88.85 AP, and F-ConV 91.74/89.95, compared with AEROBLADA at 57.13/58.00, CNNspot at 52.85/53.29, and NPR at 51.92/50.25; FatFormer reaches 89.95/87.64, Ojha 77.06/80.69, DRCT 82.53/82.28, and DIRE 52.83/52.16.
• OpenSora results, including 100 generated videos and 5,000 extracted frames: ConV scores 82.84 AUROC / 85.24 AP and F-ConV 90.16/87.38, against FatFormer at 88.76/87.99 and DRCT at 81.79/80.11. Averaged over Sora and OpenSora, ConV reaches 85.29/87.05, F-ConV 90.95/88.67, FatFormer 89.36/87.82, and AEROBLADA 56.46/60.19.
• More random transformations help: Increasing the number of applied transformations n correlates with improved detection; the experiments fix n = 20 to balance performance and efficiency, and results are averaged over five random seeds.
• Feature-space evidence supports the premise: t-SNE visualizations of DINOv2 features show natural image features overlapping substantially with those of their augmented versions, while generated images and their augmented versions separate notably — the basis for using representation similarity as the detection signal.
• Robustness to perturbations: Under JPEG compression (quality q), Gaussian blur, and Gaussian noise (both with standard deviation σ), ConV achieves the best performance in the reported comparison, and training-free methods generally degrade less than training-based ones; NPR's performance is described as degrading drastically under perturbations.
• Choice of backbone: DINOv2 variants reach AUROC values on ImageNet of 62.84 (ViT-S/14), 78.58 (ViT-B/14), 87.13 (ViT-L/14), and 85.97 (ViT-g/14); ViT-L/14 is used for the reported experiments.

Methodology in Plain English

The idea rests on a geometric picture: natural images lie on a manifold, and generated images sit slightly off it. If you nudge a natural image in a direction that stays on its manifold — a small geometric transformation, a color jitter, a blur — a model trained only on natural images barely notices, so its loss stays about the same. Do the same nudge to a generated image, and the loss changes more clearly, because the image was never on the manifold to begin with.

The authors formalize this with two functions. The first is simply the loss of a self-supervised model pre-trained on natural images (DINOv2 in the experiments); its gradient points off the manifold. The second is the same model composed with a transformation that moves along the manifold, so its gradient lies in the tangent space. Because these two gradient directions are orthogonal, the functions agree on natural images and disagree on generated images. Detection is then just a threshold on the disagreement.

Two practical touches make this efficient. First, instead of computing loss values with negative samples, they use the cosine similarity between the representation of an image and the representation of its transformed version as a surrogate. Second, they apply the transformation multiple times (20 in the experiments) and aggregate, which improves accuracy.

For F-ConV, they add a RealNVP normalizing flow on top of the DINOv2 features. The flow makes the complicated natural-image distribution look like a standard Gaussian, so deviations become easier to measure. It is trained with a shaping loss that pushes generated images away from the natural manifold and a consistency loss that amplifies the consistency gap between natural and generated images, using AdamW with a learning rate of 1e-5 and two coupling blocks. Code is available at https://github.com/tmlr-group/ConV.

Why This Matters

• Research impact: The paper reframes generated-image detection away from "collect more generated data and retrain" toward "fit the natural distribution," which sidesteps the treadmill of continuously gathering samples from every new generator. It also connects detection to the geometry of self-supervised representation spaces.
• Real-world applications:
  • Flagging deepfakes and manipulated media in political or economic contexts, which the introduction names as a sensitive misuse area.
  • Filtering synthetic images out of training corpora, since the paper cites evidence that training on generated data can degrade model quality.
  • Screening AI-generated video from closed, inaccessible models such as Sora, where the generator's weights and architecture cannot be inspected.
  • Operating in lossy, web-realistic pipelines where images have been JPEG-compressed or otherwise post-processed, thanks to the reported robustness.
• Industry relevance: A training-free detector that plugs into an existing pre-trained backbone lowers compute and data-collection costs, which matters for content platforms and dataset curators who need detection at scale and cannot feasibly retrain for each new generator release.

Future Directions

• Closing the residual gap to trained detectors: On the ImageNet average, ConV trails FatFormer (87.13 versus 93.68 AUROC) even though F-ConV edges ahead; narrowing that gap without reintroducing dependence on generated data is an open problem.
• The cost of F-ConV's training: F-ConV requires training a RealNVP flow on DINOv2 features, so the paper's shift away from training-based detection is only partial for that variant — how much generated data the flow needs, and how that scales to unseen generators, is a natural next question.
• Choice of backbone and transformation: The results depend on DINOv2, with ViT-L/14 chosen over the higher-scoring-on-ImageNet combination reported for ViT-g/14; whether other self-supervised models or transformation families change the orthogonality argument's practical value remains open (the paper points to additional pre-trained models in Appendix N).
• Threshold selection in deployment: Detection uses a threshold α on the consistency score; how that threshold should be set or adapted in a new domain, and how it behaves under adversarial post-processing designed to evade detection, is not resolved in the material summarized here.

Target Audience

Researchers and graduate students in computer vision, generative modeling, and trustworthy machine learning; practitioners building content-moderation, provenance, or dataset-curation pipelines; and readers interested in applying self-supervised pre-trained models and normalizing flows to out-of-distribution detection. Readers need comfort with manifold and gradient notation to follow the derivations, though the resulting detection procedure can be implemented by anyone familiar with PyTorch-style feature extraction.

Authors’ abstract

The increasing realism of generated images has raised significant concerns about their potential misuse, necessitating robust detection methods. Current approaches mainly rely on training binary classifiers, which depend heavily on the quantity and quality of available generated images. In this work, we propose a novel framework that exploits geometric differences between the data manifolds of natural and generated images. To exploit this difference, we employ a pair of functions engineered to yield consistent outputs for natural images but divergent outputs for generated ones, leveraging the property that their gradients reside in mutually orthogonal subspaces. This design enables a simple yet effective detection method: an image is identified as generated if a transformation along its data manifold induces a significant change in the loss value of a self-supervised model pre-trained on natural images. Further more, to address diminishing manifold disparities in advanced generative models, we leverage normalizing flows to amplify detectable differences by extruding generated images away from the natural image manifold. Extensive experiments demonstrate the efficacy of this method. Code is available at https://github.com/tmlr-group/ConV.