ImageSentinel: Protecting Visual Datasets from Unauthorized Retrieval-Augmented Image Generation

AI summary

Overview

• Research area: Computer vision and generative AI security, specifically protecting image datasets from unauthorized use inside Retrieval-Augmented Image Generation (RAIG) systems.
• Technical level: Intermediate. Readers need familiarity with retrieval-augmented generation, text-to-image models, vision-language models, and standard similarity/watermarking metrics.
• Scope: The paper proposes and empirically evaluates ImageSentinel, a protection framework that inserts synthesized "sentinel images" into a private dataset so that secret character keys can later trigger detectable generations and reveal unauthorized dataset use.

What This Paper Is About

RAIG systems boost image generation quality by pulling reference images from a database, which makes private image collections valuable and vulnerable to unauthorized incorporation. Traditional digital watermarking does not survive RAIG's feature extraction and recombination, so no reliable way exists to detect misuse. ImageSentinel addresses this by planting visually consistent sentinel images into the dataset that can be triggered by random character keys, letting a dataset owner query a suspect system and infer whether their images were used.

Key Contributions

1. The authors identify and formalize the problem of protecting visual datasets from unauthorized use in retrieval-augmented image generation systems, a setting they describe as increasingly important as these technologies spread.
2. They propose ImageSentinel, a framework that verifies dataset use through strategically crafted sentinel images, aiming for reliable detection while preserving generation quality for authorized users.
3. They introduce random character sequences (for example, "VasWiW") as retrieval keys, which the paper states ensure precise, reliable target retrieval that cannot be easily bypassed through direct generation.
4. The framework is decomposed into three components: key generation, sentinel image synthesis (attribute extraction plus key-guided synthesis), and unauthorized use detection.

Main Findings

• Superior stealthiness with GPT-4o: Sentinel images produced by GPT-4o scored higher than SDXL-generated sentinels on visual similarity to reference images on the LLaVA-Pretrain Dataset: CLIP 0.663 vs. 0.578, DINO 0.609 vs. 0.328, SigLIP 0.657 vs. 0.543, and MoCo 0.835 vs. 0.732.
• Higher triggering rates than semantic-based retrieval: Averaged over 300 samples on LLaVA-Pretrain with CLIP as retriever, ImageSentinel triggered retrieval 100.0% of the time for both SDXL and OmniGen, against 21.3% (SDXL) and 39.0% (OmniGen) for the semantic-based approach.
• Better Hit@1 retrieval accuracy: ImageSentinel reached 69.7% Hit@1, 73.8% Hit@3, and 74.6% Hit@5, compared with 58.3%, 71.7%, and 83.7% for semantic-based retrieval. The text states the Hit@1 figure as 69.67%. Semantic-based retrieval led on Hit@3 and Hit@5.
• Near-perfect detection on LLaVA-Pretrain: With 10 or 20 queries, ImageSentinel reached AUC, T@1%F, and T@10%F of 1.000 on all three tested RAIG systems (SDXL, OmniGen, GPT-4o). With 3 queries it already reached AUC 0.974 (SDXL), 0.873 (OmniGen), and 0.983 (GPT-4o).
• Baselines fail: Ward-HiDDeN and Ward-FIN stayed near random chance, with AUC values ranging roughly from 0.506 to 0.614 across query counts and RAIG systems, versus ImageSentinel's 0.873 to 1.000.
• Detection on a larger database: On the 30,000-image Product-10K test split with SDXL plus IP-adapter generation and SigLIP retrieval, a single query gave AUC 0.870, T@1%F 0.704, and T@10%F 0.754; 3 queries gave AUC 0.989; 5 queries gave AUC 0.999; 8 or more queries gave 1.000 on all three metrics. The paper concludes 3-5 queries is a practical balance of effectiveness and efficiency.
• Generation quality preserved: On LLaVA-Pretrain, original RAIG scored 0.776/0.747/0.616 (CLIP/SigLIP/DINO) with SDXL, while ImageSentinel scored 0.772/0.743/0.605 and "sentinel replacement" dropped to 0.708/0.676/0.461. OmniGen followed the same pattern (original 0.751/0.716/0.591, ImageSentinel 0.727/0.692/0.531, replacement 0.688/0.648/0.447).
• Key length matters: In an ablation with 5 queries, 6-character keys performed best (AUC 0.997, T@1%F 0.980, T@10%F 0.992), exceeding both 4-character keys (0.965, 0.848, 0.943) and 8-character keys (0.972, 0.860, 0.944). The authors attribute this to shorter keys lacking uniqueness and longer keys introducing redundant patterns.

Methodology in Plain English

The dataset owner follows a two-phase process.

Protection phase. First, unique retrieval keys are generated as random uppercase/lowercase letter strings, chosen because ordinary users rarely type such strings, so they minimally interfere with normal system operation. Next, sentinel images are synthesized in two steps: a vision-language model (GPT-4o in the main setting) analyzes randomly selected reference images from the private dataset and extracts semantic attributes and a detailed description covering subject matter, style, composition, and color palette; then a text-to-image model is given a template prompt combining that description with an instruction that the key characters "must be prominently visible while naturally integrated into the scene." The resulting sentinels are added to the private dataset to form the protected dataset, with the original images left untouched and the sentinel set much smaller than the private set. The design targets three properties: stealthiness (visual and semantic consistency with the dataset), transparency (no effect on normal generation), and triggerability (reliable activation by the keys).

Detection phase. The owner queries the suspect RAIG system with prompts built from the keys, using an instruction that strictly demands the exact key be output. Generated images are compared with the matching sentinel images using cosine similarity between normalized DINO "ViT-S/16" features. These similarities are averaged across keys to give a detection score, and if the score exceeds a threshold, the paper's decision rule concludes that the private dataset is contained in the system's reference database. The framework is framed around two hypotheses: that the generated image shows no sentinel influence, or that it exhibits sentinel characteristics.

Threat model and setup. The owner operates in a black-box setting, able to submit queries and observe outputs but not to see the reference database or generation module parameters, and able to preprocess images only before distribution. Experiments use GPT-4o as both the vision-language model and text-to-image model (compared against SDXL), three generation modules (SDXL with ViT-H IP-adapter, OmniGen, and GPT-4o), two retrievers (CLIP ViT-B/32 and SigLIP ViT-B/16), and 8 NVIDIA Tesla V100 GPUs. Evaluation uses the LLaVA-Pretrain subset of 10,000 images and the Product-10K test split of 30,000 product images, with BLIP synthetic captions for retrieval-accuracy and quality-preservation evaluation.

Why This Matters

Research impact. The paper argues that watermarking-based dataset protection, which works plausibly for text-based retrieval-augmented generation, does not transfer to images because generation involves feature extraction and recombination that destroys embedded watermarks. It reframes protection as inserting retrievable sentinel content rather than embedding signals in the data, and its results show a large empirical gap versus two watermarking baselines adapted from Ward (HiDDeN and FIN).

Real-world applications.

• E-commerce image catalogs, as demonstrated on the Product-10K dataset of 30,000 product images, where a brand could verify whether its product imagery was scraped into a third-party generation system.
• Proprietary or licensed photo and creative libraries that need a way to audit whether a commercial image generator used their collection.
• Curated visual instruction and training datasets such as the 10,000-image LLaVA-Pretrain subset, where contributors want to detect downstream misuse.
• Copyright enforcement and licensing disputes, where the detection score plus a threshold could supply evidence that a dataset was incorporated into a retrieval database.

Industry relevance. The work is directly relevant to companies operating image generation platforms, dataset marketplaces, and content-licensing businesses, where unauthorized dataset use creates legal and commercial risk. The low query cost reported (3-5 queries sufficient on a 30,000-image database) is notable for practical auditing workflows. Author affiliations with NVIDIA and Hong Kong Baptist University place the work in an industrial research context, and the authors state code is available at https://github.com/luo-ziyuan/ImageSentinel.

Future Directions

• Robustness against aware or adaptive attackers. The paper evaluates a black-box setting where the RAIG operator is not described as defending against detection. The content does not report tests against adversaries who filter, deduplicate, or perturb sentinel images, so this remains open.
• Scaling beyond the reported sizes. The authors note additional results with other database sizes are in the supplementary materials, and the main text covers 10,000 and 30,000 images. Behavior on much larger or adversarially curated databases is not characterized in the available content.
• Generalization to other RAIG designs and retrievers. Main-text detection results use CLIP as retriever on LLaVA-Pretrain and SigLIP on Product-10K; the paper defers SigLIP retriever results on LLaVA-Pretrain to the supplementary materials, leaving the generality of the findings across retrieval mechanisms partly unshown.
• Deployment trade-offs. Threshold selection, false-positive behavior in live settings, the cost of sentinel synthesis, and the risk that sentinels are detected or removed during dataset curation are not quantified in the supplied content and are natural follow-up questions.

Target Audience

This paper suits researchers and practitioners in generative AI security, dataset governance, and copyright protection who already understand retrieval-augmented generation and text-to-image pipelines. It is also useful for dataset owners, platform operators, and legal or policy teams evaluating technical mechanisms for detecting unauthorized use of image collections, and for students looking for a concrete example of how protection schemes must be redesigned when the generation pipeline changes.

Authors’ abstract

The widespread adoption of Retrieval-Augmented Image Generation (RAIG) has raised significant concerns about the unauthorized use of private image datasets. While these systems have shown remarkable capabilities in enhancing generation quality through reference images, protecting visual datasets from unauthorized use in such systems remains a challenging problem. Traditional digital watermarking approaches face limitations in RAIG systems, as the complex feature extraction and recombination processes fail to preserve watermark signals during generation. To address these challenges, we propose ImageSentinel, a novel framework for protecting visual datasets in RAIG. Our framework synthesizes sentinel images that maintain visual consistency with the original dataset. These sentinels enable protection verification through randomly generated character sequences that serve as retrieval keys. To ensure seamless integration, we leverage vision-language models to generate the sentinel images. Experimental results demonstrate that ImageSentinel effectively detects unauthorized dataset usage while preserving generation quality for authorized applications. Code is available at https://github.com/luo-ziyuan/ImageSentinel.