DRAGON: Guard LLM Unlearning in Context via Negative Detection and Reasoning

AI summary

Overview

Research area: Natural Language Processing — specifically machine unlearning in large language models (LLMs), prompt-based inference-time control, and unlearning evaluation.

Technical level: Intermediate. Readers need familiarity with LLM fine-tuning, prompting, and metrics such as ROUGE-L and multiple-choice accuracy, but the paper's core idea (detect, then reason in context) is conceptually straightforward.

Scope: The paper proposes a training-free, inference-time framework that detects prompts requiring unlearning and routes them through a chain-of-thought guard model, and it introduces new metrics for unlearning and continual unlearning.

What This Paper Is About

LLM unlearning aims to remove private or harmful knowledge from a model. Most existing methods fine-tune the model, which is expensive, often requires access to retain data, and is impractical for proprietary black-box models. DRAGON instead leaves the base model untouched: it detects whether an incoming user prompt falls into the scope of what should be forgotten, and if so, prepends generated chain-of-thought (CoT) reasoning instructions (plus retrieved safety policies) to steer the model's in-context behavior at inference time.

Key Contributions

1. A systematic, training-free unlearning framework (DRAGON) that guards deployed LLMs before inference using in-context CoT intervention, with no fine-tuning of the base model, no reliance on retain data, and applicability to black-box LLMs.
2. A pre-inference detection mechanism that identifies prompts requiring unlearning using only synthetic or paraphrased negative data. It combines a trained scoring model with similarity-based metrics (exact match, cosine similarity, BERTScore, ROUGE-L) into a unified confidence score with thresholding.
3. Three new evaluation metrics: Refusal Quality (RQ), Dynamic Deviation Score (DDS), and Dynamic Utility Score (DUS), the latter two targeting the continual unlearning setting.
4. Extensive experiments across three unlearning tasks — hazardous knowledge (WMDP), privacy records (TOFU), and copyrighted content — showing strong unlearning efficiency, preserved general language ability, and scalability across model sizes and types.

Main Findings

• Hazardous knowledge (WMDP/MMLU): On Table 1, DRAGON drives multiple-choice probability accuracy near the random-guess level of 25% for four-option questions while keeping MMLU scores essentially unchanged. For example, on Llama3.1-8B-Instruct, DRAGON reaches ProbAcc 26.2 (Biology), 23.5 (Chemistry), and 27.9 (Cybersecurity) with MMLU 68.0, versus the original model's 73.1 / 54.9 / 46.7 and MMLU 68.0. DRAGON also achieves the highest Refusal Quality on WMDP (e.g., 0.921 Biology, 0.795 Chemistry, 0.875 Cybersecurity on Llama3.1-8B-Instruct; 1.296 / 1.149 / 1.183 on Mixtral-8x7B-Instruct (47B)).
• Privacy records (TOFU): On Llama2-7B-Chat (Table 2), DRAGON records the best deviation scores of 21.4 (TOFU-1%), 23.1 (TOFU-5%), and 26.5 (TOFU-10%), retains the highest model utility at 0.6337 across all three splits, and leads on KFR/KRR (e.g., 0.98 / 0.88 at 1%, 0.99 / 0.87 at 5%, 1.00 / 0.90 at 10%). Fine-tuning baselines such as GA, KL, and DPO collapse to MU 0.0 at the 5% and 10% forgetting levels.
• Continual unlearning: Under three sequential forget sets (forget01, forget05, forget10), DRAGON achieves the best DDS and DUS on both Llama2-7B-Chat (DDS 0.2494, DUS 1.0) and Phi-1.5B (DDS 0.2853, DUS 1.0), versus baselines such as GA (0.9351 / 0.6836) and KL (0.9629 / 0.6855) on Llama2-7B-Chat. The paper notes ICUL+'s DUS of 1.0 is expected because it operates with full access to all forget data.
• CoT instruction matters: Removing CoT degrades performance (DS 43.9 / 40.9 / 39.9 versus 21.4 / 23.1 / 26.5 for full DRAGON on Llama2-7B-Chat across the three TOFU splits). Static template refusal (Guardrail+) produces a consistency-score gap of 0.44 versus just 0.01 for DRAGON, indicating far weaker contextual alignment.
• Detection robustness: The proposed detector achieves 100.0% accuracy on all three TOFU forget splits and 98.9 / 98.3 / 96.7 on WMDP-bio / chem / cyber, outperforming or matching a RoBERTa-based classifier (100.0 / 100.0 / 100.0, 84.2 / 78.2 / 79.4) and a GPT-4o-based classifier (95.0 / 97.5 / 92.2, 93.1 / 100.0 / 97.5).
• Sensitivity: Evaluated across Qwen2.5 sizes [1.5B, 3B, 7B, 32B] and base versus instruct variants, larger models generally perform better and instruct variants consistently outperform base counterparts. The method was also tested on GPT-4o, Llama-4, and Llama-3.1-70B-Instruct.
• Scale claim: The paper states DRAGON delivers its strongest results on more capable large models and incurs no additional cost when scaling to larger models.
• Note on model count: The paper text states DRAGON "consistently achieves the best unlearning performance across nine LLMs," while the Table 1 caption describes four LLMs (Zephyr-7B, Llama3.1-8B-Instruct, Yi-34B-Chat, Mixtral-8x7B-Instruct (47B)). Additional results are referenced in Table 12.
• Copyrighted content unlearning: Referenced as a third task (Table 13), but the results are not included in the provided content.

Methodology in Plain English

1. Build an unlearn store. When an unlearning request arrives, a locally deployed Llama3.1-70B-Instruct generates four rephrased candidates of the forget prompt. Rejection sampling based on BERTScore keeps the most semantically similar candidate. Only paraphrased prompts and embeddings are stored — never the original completions — to reduce leakage risk.
2. Detect whether an incoming query needs unlearning. For privacy records, the confidence score combines an exact name match plus the maximum cosine similarity to stored prompts. For harmful concepts, a Llama-3.1-7B-Instruct scoring model fine-tuned on synthetic harmful and benign queries produces a probability, with BERTScore and ROUGE-L similarity checks as secondary validation. If the combined score exceeds a threshold, the query is flagged.
3. Generate CoT instructions. A guard model (Llama3.1-8B-Instruct fine-tuned on a CoT dataset) produces step-by-step reasoning instructions conditioned on the query and the retrieved safety policy. The CoT dataset was built with GPT-4o: 800 synthetic questions about fictitious authors plus 200 paraphrased TOFU questions, with rejection sampling for quality.
4. Intervene in context. The generated reasoning and policy are prepended to the input so the base LLM refuses or redirects without any weight updates. For TOFU, a "double protection" strategy fabricates synthetic author information; for WMDP, relevant policies and refusal guidelines are extracted and enforced.
5. Evaluate with new metrics. RQ combines refusal-template similarity, classifier-estimated refusal rate, and a gibberish-detector quality score. DDS averages trade-off scores over T steps and penalizes upward deviations, with β = 0.5. DUS measures 1 minus the average absolute utility change across steps.

Why This Matters

Unlearning is tied to legal compliance (the paper cites GDPR's requirement to remove user data on request) and to preventing the spread of harmful knowledge. Because DRAGON never touches model weights and never needs retain data, it sidesteps the two biggest practical barriers to deployment: the cost of gradient-based optimization over millions to billions of parameters, and the frequent unavailability of original training data due to privacy restrictions, expired licenses, or intellectual property concerns.

Real-world applications:

• Privacy compliance: Removing personal records from deployed assistants on request, as modeled by the TOFU privacy-record task.
• Harmful knowledge removal: Suppressing hazardous knowledge in biology, chemistry, and cybersecurity, as modeled by WMDP, relevant to biosecurity and cyber-risk governance.
• Copyrighted content removal: The paper lists copyrighted content unlearning as one of its three evaluation tasks.
• Continual, dynamic deployments: Handling unlearning requests that arrive

Authors’ abstract

Unlearning in Large Language Models (LLMs) is crucial for protecting private data and removing harmful knowledge. Most existing approaches rely on fine-tuning to balance unlearning efficiency with general language capabilities. However, these methods typically require training or access to retain data, which is often unavailable in real world scenarios. Although these methods can perform well when both forget and retain data are available, few works have demonstrated equivalent capability in more practical, data-limited scenarios. To overcome these limitations, we propose Detect-Reasoning Augmented GeneratiON (DRAGON), a systematic, reasoning-based framework that utilizes in-context chain-of-thought (CoT) instructions to guard deployed LLMs before inference. Instead of modifying the base model, DRAGON leverages the inherent instruction-following ability of LLMs and introduces a lightweight detection module to identify forget-worthy prompts without any retain data. These are then routed through a dedicated CoT guard model to enforce safe and accurate in-context intervention. To robustly evaluate unlearning performance, we introduce novel metrics for unlearning performance and the continual unlearning setting. Extensive experiments across three representative unlearning tasks validate the effectiveness of DRAGON, demonstrating its strong unlearning capability, scalability, and applicability in practical scenarios.