Prompt-Based Safety Guidance Is Ineffective for Unlearned Text-to-Image Diffusion Models

AI summary

Overview

Research area: AI safety and content moderation for text-to-image diffusion models (machine learning / generative models).

Technical level: Intermediate. The paper assumes familiarity with diffusion sampling, classifier-free guidance, negative prompts, and textual inversion, though the core idea is conceptually simple.

Scope: This paper diagnoses an incompatibility between two families of safety mechanisms for text-to-image models — weight-modifying "unlearning" and inference-time "training-free" guidance — and proposes swapping hand-written negative prompts for embeddings recovered through concept inversion.

What This Paper Is About

Text-to-image diffusion models can be pushed into generating harmful content by malicious prompts, and the field has produced two separate defenses: fine-tuning weights so the model "unlearns" harmful concepts (training-based), and steering generation at inference time using negative prompts (training-free). The authors observe that combining these two approaches — a natural thing to do — often yields only marginal improvement or even worse defense, because an unlearned model has been trained to ignore exactly the explicit negative prompts that the training-free methods depend on. This paper proposes replacing those explicit negative prompts with implicit "concept embeddings" recovered by inverting harmful images back into the text embedding space, restoring the effectiveness of the training-free methods on top of unlearned models.

Key Contributions

1. Identifies a fundamental incompatibility: The authors show that training-free guidance methods built on explicit negative prompts (SLD, SAFREE) largely fail, and can underperform the unlearned base model alone, when applied to a model that has already been unlearned (DUO).
2. Proposes a drop-in replacement: They replace the prompt-based negative embeddings in the training-free methods with implicit negative concept embeddings obtained via Concept Inversion, requiring no modification to either the unlearning method or the guidance method.
3. Demonstrates consistent DSR improvements under equivalent prior preservation: On both violence and nudity benchmarks and across four DUO regularization settings, the proposed method moves the defense–preservation trade-off curve toward the upper-right region.
4. Shows transferability of extracted embeddings: Concept embeddings extracted from one checkpoint (β=500) remain effective when reused across other checkpoints (β=250, 1000, 2000), suggesting unlearned models retain a residual negative text embedding space shared with their base model.

Main Findings

• Training-free methods degrade on unlearned models: The paper reports that combining SLD or SAFREE with the unlearned DUO model produces only marginal improvements on the violence task and degradations on the nudity task, supporting the claim that unlearned models stop responding to explicit negative prompts.
• Nudity results (Table 2): At β=2000, DUO alone achieves DSR 0.4421 with PP 0.8021, while DUO+SLD+Ours reaches DSR 0.9368 with PP 0.7119 and DUO+SAFREE+Ours reaches DSR 0.9474 with PP 0.6318. At β=1000, DUO+SLD+Ours reaches DSR 0.9895 versus 0.7158 for both DUO and DUO+SLD.
• Violence results (Table 1): At β=1000, DUO+SAFREE+Ours achieves DSR 0.9467 and PP 0.6763, compared with DSR 0.7933 and PP 0.7332 for DUO+SAFREE, and DSR 0.6133 and PP 0.8204 for DUO alone.
• Improved trade-off position: For the same PP level, the proposed method consistently yields a higher DSR than the baselines on both tasks, which the authors interpret as evidence that unlearned models still store implicit latent embeddings related to harmful content.
• Embedding transferability: A concept embedding extracted at β=500, when reused on other checkpoints, gives nudity DSR of 1 at β=250, 0.9895 at β=1000, and 0.9053 at β=2000 under DUO+SLD+Ours (Table 3).
• Base-model sensitivity: Prompting the original base model (SD v1.4) with the extracted concept embedding still generates a harmful image, indicating the unlearned model shares residual negative text embedding space with its base model.

Methodology in Plain English

The authors start from a simple observation made on existing checkpoints: a model that has been fine-tuned to forget a harmful concept no longer reacts to words like "Sexual Acts" or "with a gun" the way the original model did, so guidance methods that subtract such words have little to subtract from.

Their fix is to stop describing the harmful concept in words and instead find the vector in the model's text embedding space that actually corresponds to it. To obtain that vector, they use Concept Inversion, a diffusion-based inversion technique built on Textual Inversion: given a set of harmful images, they freeze the model and optimize a single embedding vector so that the model would have generated those images from it. That optimized vector is stored as a special token embedding and then handed to the existing guidance methods in place of the manually written negative prompt list.

In the experiments they set the number of inverted concepts to 1 for simplicity. They build on DUO as the unlearned model, sample with DDIM over 50 steps, and test four DUO checkpoints determined by the regularization hyperparameter β ∈ {250, 500, 1000, 2000}; smaller β means safer generation but a larger deviation from the base model. Nudity is evaluated on 95 adversarial prompts from Ring-a-Bell, and violence on 150 prompts from the I2P benchmark filtered to a Q16 percentage of 0.95 or higher. Success is measured two ways: Defense success rate using the NudeNet detector for nudity and the Q16 classifier for violence, and Prior Preservation measured as the average of 1-LPIPS against images from the original SD v1.4 model.

For the inversion data, nudity uses SD v1.4 generations from the I2P benchmark category "sexual," filtered to images NudeNet scores at 0.75 or higher, leaving 77 images. Violence uses 150 images generated from I2P prompts with Q16 of 0.95 or higher that are not used in evaluation. Inversion is run with Adam, a learning rate of 5×10⁻³, batch size 1, and 3000 gradient steps, taking roughly 15 minutes per unlearned model on an NVIDIA GeForce RTX 3090 with CUDA 11.4.

Why This Matters

Impact on research: The paper reframes training-based unlearning and training-free guidance as not simply orthogonal but actively interfering, and shows a minimal intervention that partially repairs the combination. It also raises the point that unlearning leaves recoverable structure in the embedding space, which is relevant to how unlearning success is measured and reported.

Real-world applications:

• Content moderation pipelines that layer inference-time filters on top of models already fine-tuned for safety, where the filters currently lose effectiveness.
• Deployment of already-released unlearned checkpoints, where retraining is infeasible but additional post-hoc safety control is possible.
• Media and creative tooling that needs prompt semantics preserved while suppressing specific harmful categories, since PP is tracked as a first-class metric.
• Post-unlearning auditing: checking whether a supposedly erased concept is still recoverable through an inverted embedding.

Industry relevance: The method requires no architectural change and integrates into existing SLD and SAFREE codebases, with a per-model cost the authors describe as not computationally intensive (about 15 minutes of inversion). That makes it practical for teams maintaining safety wrappers around open or legacy diffusion checkpoints.

Future Directions

• Reducing per-model cost: The paper states that the concept embedding must currently be extracted separately for each unlearned model and requires access to a dataset of harmful images — both are stated limitations.
• Extending transferability into a general recipe: The authors show cross-checkpoint transfer works in their setting and highlight transferring embeddings between unlearned models that share the same base model, but do not establish how far this generalizes.
• Exploring K* > 1: The experiments fix K* = 1 for simplicity, leaving open how multiple inverted concepts for different harmful categories combine.
• Re-examining the training-free / training-based boundary: The paper frames the incompatibility as a motivation for a new direction in post-unlearning safety control, but does not propose a mechanism that avoids the incompatibility in the first place.

Target Audience

Researchers and engineers working on safety, unlearning, and inference-time control for diffusion models will get the most from this paper, particularly those who combine multiple safety mechanisms and assume their effects add up. It is also relevant to practitioners maintaining deployed or released checkpoints who need post-hoc mitigation, and to readers interested in the dual-use nature of textual inversion techniques.

Authors’ abstract

Recent advances in text-to-image generative models have raised concerns about their potential to produce harmful content when provided with malicious input text prompts. To address this issue, two main approaches have emerged: (1) fine-tuning the model to unlearn harmful concepts and (2) training-free guidance methods that leverage negative prompts. However, we observe that combining these two orthogonal approaches often leads to marginal or even degraded defense performance. This observation indicates a critical incompatibility between two paradigms, which hinders their combined effectiveness. In this work, we address this issue by proposing a conceptually simple yet experimentally robust method: replacing the negative prompts used in training-free methods with implicit negative embeddings obtained through concept inversion. Our method requires no modification to either approach and can be easily integrated into existing pipelines. We experimentally validate its effectiveness on nudity and violence benchmarks, demonstrating consistent improvements in defense success rate while preserving the core semantics of input prompts.