Membership Inference Attacks Against Fine-tuned Diffusion Language Models

AI summary

Overview

Research area: Privacy and security of machine learning, specifically membership inference attacks (MIAs) against diffusion language models (DLMs).

Technical level: Advanced. The paper assumes familiarity with language model architectures, loss-based attack design, robust statistics, and MIA evaluation metrics such as AUC and TPR at low false positive rates.

Scope: The paper introduces and evaluates Sama, a membership inference attack framework designed specifically for fine-tuned diffusion language models, tested on two DLMs across nine datasets.

What This Paper Is About

Membership inference attacks ask whether a specific piece of text was used to train a model. A large body of work has studied this question for autoregressive language models, which predict tokens one at a time from left to right. Diffusion language models, which instead reconstruct many masked tokens at once using bidirectional context, had not been systematically tested for this kind of privacy leakage. The paper's goal is to show how the DLM masking mechanism creates new, multiple probing opportunities for an attacker, and to build a practical attack that exploits them.

Key Contributions

1. Formal analysis of how DLM masking changes the MIA problem. The paper contrasts the single fixed signal available in autoregressive models (one context configuration per token position) with the exponentially larger space of mask configurations in DLMs, and argues this provides many more independent chances to detect memorization.

2. The Sama attack framework. Sama (Subset-Aggregated Membership Attack) combines progressive masking across increasing densities, local random subset sampling, sign-based aggregation robust to heavy-tailed noise, and inverse-step weighting that prioritizes sparser masks.

3. Empirical demonstration on state-of-the-art DLMs. Two models, LLaDA-8B-Base and Dream-v0-7B-Base (also referred to as Dream-v0-Base-7B), are fine-tuned on member datasets from six MIMIR domain splits plus WikiText-103, AG News, and XSum, and evaluated against twelve MIA baselines.

4. Reported attack gains. The paper reports that Sama achieves a 30 percent relative AUC improvement over the best baseline, with up to 8 times improvement at low false positive rates.

Main Findings

• Baseline MIA methods transfer poorly to DLMs. In Table 1, most autoregressive and image-diffusion baselines sit near chance on several datasets. For example, on ArXiv, Loss scores AUC 0.506, Min-K% 0.488, and SecMI 0.520, while the strongest baseline, Ratio, reaches 0.597.

• Sama substantially outperforms all baselines. On ArXiv, Sama achieves AUC 0.850 with TPR@10% FPR of 0.586, TPR@1% FPR of 0.178, and TPR@0.1% FPR of 0.014, compared to Ratio at 0.597 AUC and 0.181 TPR@10% FPR.

• Gains are largest at strict false positive rates. On GitHub, Sama reaches AUC 0.876, TPR@10% FPR 0.647, TPR@1% FPR 0.259, and TPR@0.1% FPR 0.075, versus Ratio at 0.743 AUC, 0.355 TPR@10% FPR, and 0.081 TPR@1% FPR.

• Performance varies by domain. Sama's AUC is 0.850 on ArXiv and 0.876 on GitHub, but 0.657 on HackerNews. The table in the provided content is truncated after the Neighbor row for PubMed Central, Wikipedia (en), and Pile CC, so Sama's scores on those three datasets are not shown here.

• Membership signals in DLMs are sparse and configuration-dependent. In a controlled ArXiv experiment using 100 distinct random mask configurations per sample, the standard deviation of signal strength caused purely by changing the mask configuration (σ ≈ 0.10) exceeded the average member/non-member margin (δ ≈ 0.06).

• Member and non-member signal distributions overlap heavily. Aggregate densities show non-member signals centered near zero while member signals shift positive with a heavier right tail, but with substantial overlap.

• Sign-based aggregation is theoretically motivated. The paper grounds the binary indicator approach in the Hodges-Lehmann theorem and the robust statistics literature, noting that for non-members the indicator equals 1 with probability exactly 0.5 regardless of the noise distribution's properties, even under infinite variance.

• Fine-tuned models retain utility. The paper states the fine-tuned models maintain strong utility, with details in Section D.2.

• Overhead is reported as low. The paper states in Appendix D.7 that the robust aggregation logic incurs negligible computational overhead compared to standard query-based baselines.

Methodology in Plain English

The attack works by comparing a fine-tuned target model against the pre-trained base model it came from. For a given piece of text, the attacker repeatedly hides different parts of it with mask tokens, asking both models to guess the hidden words. If the text was in the fine-tuning data, the fine-tuned model tends to be more confident than the base model on those hidden positions.

The key difficulty is that most mask patterns produce no useful signal. The paper's observation is that only some mask configurations, particularly sparse ones where most context remains visible, trigger memorization. To handle this, Sama does three things.

First, it masks tokens progressively: it starts by hiding about 5 percent of the tokens and gradually increases to about 50 percent over 16 steps, using 128 sampled subsets of 10 tokens at each step. Second, instead of averaging raw loss differences, it checks only the sign of each difference, counting how often the base model had higher loss than the fine-tuned model. This binary decision is robust to a few extreme outlier tokens that would otherwise dominate an average. Third, it weights the results from each density level inversely by step number, giving more influence to the sparse, cleaner early steps than to the dense, noisier later steps. The final membership score is a weighted sum of these per-step fractions, ranging from 0 to 1.

Why This Matters

Impact on research. This is presented as the first systematic MIA study on diffusion language models. It establishes that privacy conclusions drawn from autoregressive models do not automatically carry over to the diffusion paradigm, and it introduces a masking-aware attack formulation that other researchers can build on or defend against.

Real-world applications:

• Auditing released model weights. When organizations publish fine-tuned DLM weights alongside their base models, this attack gives auditors a concrete method to test whether private fine-tuning data can be extracted or attributed.

• Regulatory and compliance review. Teams assessing whether a deployed model memorizes training data can use these results when building evidence for data-protection obligations.

• Training data governance. Organizations deciding what data to include in DLM fine-tuning can gauge the risk that specific records become detectable.

• Model serving interfaces. The threats apply to DLM services that expose in-filling or masked-input APIs, which the paper notes already exist, such as LLaDA's demo.

Industry relevance. The paper connects to commercial diffusion language model development, citing Google's Gemini Diffusion as evidence the paradigm is moving toward production. The threat model assumes grey-box access consistent with open-weight release practices, meaning the attack is realistic in current deployment patterns rather than requiring an unrealistic level of access.

Future Directions

• Designing tailored defenses for DLMs. The paper explicitly calls for privacy defenses built for the diffusion paradigm, noting that existing protections derived from autoregressive models may not be adequate.

• Extending to broader model families and scales. Sama was validated on two DLMs (LLaDA-8B-Base and Dream-v0-7B-Base); whether the findings generalize to other diffusion architectures is not established.

• Understanding reference model dependence. The paper defers to Section D.6 the question of what happens when the ideal reference, the original pre-trained base model, is unavailable and an alternative must be used.

• Characterizing which masking patterns trigger memorization. The controlled ArXiv experiment shows high intra-sample variance across 100 mask configurations, but the paper does not identify which specific configurations reliably activate memorization, leaving a question about predictability and predictability-based defenses.

Target Audience

This paper is most useful to privacy and security researchers working on membership inference and training-data memorization, engineers and safety teams deploying or releasing fine-tuned diffusion language models, and policy or compliance professionals who need to understand emerging privacy risks in non-autoregressive language model architectures. Readers will benefit from prior familiarity with language model training objectives and standard MIA evaluation metrics.

Authors’ abstract

Diffusion Language Models (DLMs) represent a promising alternative to autoregressive language models, using bidirectional masked token prediction. Yet their susceptibility to privacy leakage via Membership Inference Attacks (MIA) remains critically underexplored. This paper presents the first systematic investigation of MIA vulnerabilities in DLMs. Unlike the autoregressive models' single fixed prediction pattern, DLMs' multiple maskable configurations exponentially increase attack opportunities. This ability to probe many independent masks dramatically improves detection chances. To exploit this, we introduce SAMA (Subset-Aggregated Membership Attack), which addresses the sparse signal challenge through robust aggregation. SAMA samples masked subsets across progressive densities and applies sign-based statistics that remain effective despite heavy-tailed noise. Through inverse-weighted aggregation prioritizing sparse masks' cleaner signals, SAMA transforms sparse memorization detection into a robust voting mechanism. Experiments on nine datasets show SAMA achieves 30% relative AUC improvement over the best baseline, with up to 8 times improvement at low false positive rates. These findings reveal significant, previously unknown vulnerabilities in DLMs, necessitating the development of tailored privacy defenses.