ASAP: Visual Analytics for Identifying and Analyzing Image Patterns in AI-generated Images

AI summary

Overview

• Research area: Human-Computer Interaction (cs.HC), specifically visual analytics and human-in-the-loop tooling for detecting and explaining AI-generated ("deepfake") images.
• Technical level: Advanced. The backend relies on CLIP embedding surgery, orthogonal projections, transformer attention-gradient relevance propagation, and a custom normalized contribution metric, though the frontend is designed for users without formal ML training.
• Scope: The paper presents ASAP (Analyzing and Summarizing AI-generated image Patterns), a design study combining an interpretability-focused fake-image detector with a four-view visual analytics dashboard, validated through a user study and two application scenarios on established fake image detection benchmarks.

What This Paper Is About

Automated fake-image detectors are increasingly accurate but typically behave as black boxes: they output a real/fake label without explaining which pixels drove the decision, they often fail when confronted with images from unfamiliar generative models, and their outputs are hard to act on. The authors argue this gap matters more now because the people doing deepfake analysis (digital forensic analysts, misinformation researchers, journalists, moderators, legal experts) are computationally literate but not ML engineers, and they need to reason about why an image looks real or fake. ASAP addresses this by building a detector whose internal representation is deliberately interpretable, then exposing that representation through coordinated visual views that let users find, quantify, and compare deceptive pixel patterns across collections of authentic and AI-generated images.

Key Contributions

1. An interpretable CLIP-adapted image encoder. The pipeline chains CLIP:ViT-B/32 (512-dimensional output), a "forget-to-spell" orthogonal projection that strips text-related information (yielding 256 dimensions), a distiller layer that reduces this to 16 orthogonal authenticity dimensions, and a sigmoid classification head for the real/fake prediction.

2. Dimension-specific pixel relevance masks. Rather than a single saliency map from a final logit, the method backpropagates gradients from each of the 16 distilled dimensions through CLIP's transformer attention maps, producing 16 distinct masks per image that highlight different influential pixel groups.

3. A uniform, cross-image comparable pattern contribution metric. A normalized scalar in the range [-1, 1] combines each dimension's image-specific value with the classifier's global weight; its sign indicates direction (positive for fake, negative for real) and its magnitude indicates influence strength, enabling comparison of pixel-group influence across images, groups, and outcomes.

4. A coordinated visual analytics dashboard. Four linked views (Representation Overview, Image View, Dimension View, Pattern View) support an overview-to-detail workflow, custom cell glyphs summarizing classifier specificity and sensitivity, concept clustering of recurring patterns, and cell-level annotation for incremental analysis.

Main Findings

• Interpretability can be built into detection rather than bolted on. The authors state the classifier achieves comparable detection performance to state-of-the-art baselines (Ojha et al.) on benchmark datasets, but explicitly note the benchmark numbers are relegated to the Appendix due to page constraints and are not reported in the main text. The stated emphasis is that competitive accuracy comes together with interpretability, which black-box detectors do not provide.

• Text supervision in CLIP actively harms pixel-level fake analysis. The paper argues CLIP's contrastive training entangles textual information in its visual embeddings, which compromises gradient-based identification of non-semantic pixels. The forget-to-spell orthogonal projection is introduced specifically to remove this.

• Sixteen orthogonal dimensions yield diverse, separable explanations. Orthogonality among the distilled dimensions means the 16 propagated gradient maps capture pixel importance from distinct perspectives, isolating individual regions instead of producing one aggregated saliency map, and enabling fine-grained examination of subtle artificiality indicators.

• A worked example links a specific facial feature to detector sensitivity. In the paper's Figure 4 example, a user compared dimension contributions between two "brown/black horse head" cells, one with severe sensitivity and one with minor sensitivity. Dimension 2 differed notably between them, and pixel groups linked to dimension 2 correlated strongly with pixels near the horse's eyes. Severe-sensitivity cell images had high dimension 2 values while minor-sensitivity images had low values, suggesting the portrayal of horse eyes significantly influences detection.

• Evaluation spans a user study and two application scenarios. The abstract and introduction state the system was evaluated through a mixed-methods user study and two application scenarios using established fake image detection benchmarks. The truncated content does not report participant counts, study tasks, or quantitative study outcomes.

• Three named challenges motivate the design. The paper frames existing deepfake approaches as limited by generalizability (distinct visual signatures across GANs vs. diffusion models mean detectors fail across models), interpretability (black-box predictions with no explanation), and actionability.

Methodology in Plain English

The researchers trained two binary real-vs-fake classifiers, one per generative paradigm, so that each could serve as an interpretable encoder for its model type.

Data. For GANs, they used the proGAN dataset: authentic images from the PASCAL object detection challenge plus fake counterparts from proGAN, a balanced set of 30,000 real and 30,000 fake images per class across 20 classes. For diffusion, they used the DetectingSyntheticImage dataset, specifically 20,000 fake human face images generated by latent diffusion models (LDMs), paired with real faces from the Flickr-Faces-HQ (FFHQ) dataset, which contains 70,000 high-quality human face images.

Building the encoder. Each image first goes through the frozen CLIP:ViT-B/32 image encoder, producing a 512-dimensional generic visual feature vector. A "forget-to-spell" orthogonal projection then removes text-related content, using an orthogonality loss trained on a diverse dataset including natural images, text-only images, and images with overlaid text, producing a 256-dimensional visually focused vector. A two-layer linear classifier follows: a distiller layer mapping 256 dimensions down to 16 orthogonal dimensions, and a classification head applying a sigmoid. The loss combines binary cross-entropy (λ_BCE = 3) with an orthogonality penalty on the distiller weights (λ_ortho = 1), where the penalty measures deviation of W·Wᵀ from the identity. Training used the Adam optimizer with batch size 32 and learning rate 1e-3 for 12 epochs, completing in 68 seconds on a MacBook Pro with an M1 chip. The authors note they empirically assessed 16 as a good balance between performance and explainability, and that ASAP can support other dimensionalities.

Finding influential pixels. The image is split into a 7×7 grid of patches (k = 224/32 = 7), yielding 49 tokens plus a [CLS] token, each of dimension 512. Relevance is computed by taking gradients of the attention maps in the last transformer block, applying a Hadamard product with the attention map itself, averaging over heads, and keeping only positive values. Because this is done separately for each of the 16 distilled dimensions, each image produces 16 relevance maps, resized to image dimensions and normalized to [0, 1] to serve as masks.

Quantifying pattern influence. For each image, each pixel group is tied to one distilled dimension value v_i and the corresponding classification-head weight w_i. The product v_i × w_i is normalized by dividing by the sum of absolute products across all 16 dimensions, preserving the sign, yielding a contribution score between -1 and 1.

Revealing recurring patterns. To surface shared concepts across similar images, the authors use the relevance maps as masks to extract segments from images whose distilled representations are close in L2 distance, discard overlapping segments using Intersection over Union, encode the remaining segments through the full encoder, and cluster them with k-means into three groups (empirically chosen based on the number of images per cell).

Interface. The dashboard is built on a Flask backend handling embeddings, segmentation, and influence measurement, with a React and d3.js frontend. Users begin by selecting a generative model and subject (e.g., real versus fake "horses" from proGAN). The Representation Overview places images in a 2D grid using IsoMatch to preserve pairwise distances, divided into a user-definable m×m grid of cells (m defaults to 30), each rendered as a custom glyph. The glyph is a pie chart with three radial layers and four sectors representing true positives, true negatives, false positives, and false negatives; color coding is blue for real images, orange for fake, green for correct predictions, red for incorrect, with saturation indicating classifier confidence. Selecting a cell (highlighted with a gold square) loads its images into the Image View, where images are again arranged by IsoMatch and annotated with a top-right circle for ground truth and a drop shadow for the prediction. A "Show Concept" feature opens a Concept View popup, and lassoing images loads them into the Pattern View, which lays out one row per image across 16 columns corresponding to the 16 dimensions, with each cell overlaid by a pixel relevance heat map.

Why This Matters

The paper targets a widening audience of non-ML practitioners who must explain image authenticity as part of their jobs, and argues that interpretability and actionability are as important as raw detection accuracy in that context. Its distinctive move is linking collection-level summaries of image sets directly to the specific pixel patterns responsible for a detector's decisions, and treating the detector's embedding space itself as the explanation substrate rather than applying post-hoc explanation to a closed model.

Real-world applications:

• Misinformation and fact-checking workflows: helping researchers and journalists trace which visual features caused an image to pass or fail detection, and document recurring deceptive patterns.
• Digital forensics and legal review: producing evidence about specific image regions that contributed to an authenticity judgment, rather than an opaque score.
• Social media content moderation: prioritizing suspicious cells near the decision boundary where fake images are most often misclassified as real.
• Generative model auditing and comparison: comparing patterns across GAN- and diffusion-generated images to see how artificiality signatures differ between model families.

Industry relevance: The system's modular design targets adaptability as generative models churn, a stated requirement given that detectors effective for one model can fail for others. Model developers, platform trust-and-safety teams, and vendors building provenance or authenticity tooling are the likely beneficiaries of a detector whose explanations can be reviewed by humans.

Future Directions

• Investigate dimensional space tradeoffs. The authors explicitly state that ASAP can support different dimensional spaces and that they plan to study tradeoffs in larger and smaller spaces as future work.
• Broaden and formalize generalization testing. The system covers GANs and diffusion models; the design challenge of adapting to new models as they emerge remains a stated open requirement of the modular approach.
• Design guidance for pattern-heavy interfaces. The paper notes there are currently no established guidelines for designing an interface for this workflow, and that the sheer number of potential patterns risks overwhelming users.
• Extend the paradigm beyond binary real/fake. Related work is cited on human- versus AI-generated paintings, and the paper frames ASAP as addressing an analytic gap beyond simple fake/real labeling, leaving room for richer authenticity analysis.

Target Audience

This paper is most useful to visual analytics and HCI researchers building human-in-the-loop tools for generative AI, and to explainable-AI researchers interested in embedding-level rather than post-hoc interpretability. It also serves ML practitioners working on deepfake detection who want detector internals that produce human-readable pixel-group explanations, and applied practitioners (forensic analysts, misinformation researchers, moderators, journalists) who need to understand what an authenticity model is reacting to. Readers without a background in transformer attention, CLIP embeddings, or projection-based representation learning will find the backend sections demanding; the interface section is accessible to a general technical reader.

Authors’ abstract

Generative image models can produce highly realistic images, raising concerns about potential misuse in creating deceptive content. Current deepfake approaches face several challenges, including limited generalizability, lack of interpretability, and poor actionability. To help address these, we present ASAP, an interactive visualization system designed to empower users in the analysis and summarization of deceptive patterns in AI-generated images. ASAP introduces a novel CLIP-adapted image encoder that generates interpretable representations, enabling the extraction of influential pixel regions via calculated masks. This approach facilitates the identification of key deceptive features through influence measurement techniques. These backend techniques are integrated into a visual analytics dashboard that allows users to quantify and analyze authenticity-indicative patterns in image collections containing both authentic and AI-generated images. This approach also supports the comparative analysis of various generative models, including GANs and diffusion models. We demonstrate ASAP's efficacy through a user study and two application scenarios using established fake image detection benchmarks, showcasing its ability to effectively extract and quantify deceptive patterns.