LR-DWM: Efficient Watermarking for Diffusion Language Models

AI summary

Overview

Research area: Natural language processing — watermarking of text generated by diffusion language models (DLMs).

Technical level: Intermediate. The paper assumes familiarity with language-model decoding, logits, hashing-based watermarking, and diffusion-style iterative denoising, but its core idea is describable in plain terms.

Scope: This paper introduces LR-DWM, a two-sided (left-and-right neighbor) watermarking scheme for diffusion language models, and evaluates it on LLaDA-8B-Instruct and DREAM-7B-Instruct against the vanilla models and two prior DLM watermarking methods.

What This Paper Is About

Most watermarking methods for large language models assume tokens are generated one at a time in a fixed left-to-right order, so each new token can be biased using the tokens already generated before it. Diffusion language models break that assumption: they iteratively denoise an entire sequence, and tokens are finalized in an order that is not known in advance, so the standard autoregressive watermarking toolkit does not directly apply. The paper's goal is a watermark that works under this non-sequential decoding while adding almost no runtime or memory cost.

Key Contributions

1. A two-sided watermarking method for diffusion language models. LR-DWM biases each token using both its left and right neighbors, independently, whenever either is available, rather than restricting the watermark constraint to a single causal direction.
2. Order-agnostic embedding and detection. The watermark is defined as a set of local constraints over the final token sequence, using only previously determined neighboring tokens, so detection does not require inverting the hashing process or knowing the denoising schedule.
3. Efficient watermarking for DLMs. The paper shows LR-DWM stays close to the non-watermarked baseline in wall-clock generation time and peak GPU memory, in contrast to methods that incur substantial per-step computation or that cache large lookup tables.
4. Competitive detectability and text quality. The work reports detection rates and perplexity on two diffusion models, plus robustness under standard non-adaptive text perturbations.

Main Findings

• Detectability-quality trade-off on LLaDA. At fixed detection rates of 90%, 99%, and 99.5%, LR-DWM perplexity (± SEM) is 2.80 ± 0.46, 3.32 ± 0.65, and 3.37 ± 0.66, compared with DMARK at 2.82 ± 0.51, 3.28 ± 0.61, and 3.34 ± 0.63, and WM-DLM at 5.07 ± 1.40, 6.14 ± 1.84, and 6.33 ± 1.90.
• Minimal computational overhead. LR-DWM remains close to the non-watermarked deterministic baseline in both wall-clock time and peak GPU memory on a single NVIDIA H100.
• DMARK's memory cost. DMARK nearly doubles peak GPU memory consumption because it caches the entire hash table in advance.
• WM-DLM's runtime cost. WM-DLM shows higher runtime overhead, reflecting the cost of expectation-based scoring under stochastic decoding; its performance also depends strongly on stochastic decoding with high temperature, which the authors note degrades the base model it watermarks.
• Statistical calibration. Using 10,000 human-written texts of length 400 tokens from the C4 corpus, the authors set a Z-score threshold empirically corresponding to 1% FPR and confirm a 1% error rate on that human-written corpus.
• Robustness under non-adaptive attacks. At a fixed operating point achieving 100% detection on clean watermarked text with δ = 3.25 and an average Z-score of 6.653, LR-DWM retains 98.83% detection under 10% word deletion, 98.25% under 10% random word substitution, 99.42% under 10% BERT-based context-aware substitution, 55.56% under English-to-Chinese-to-English back-translation, and 15.79% under paraphrasing with Qwen-2B.
• Paraphrasing is the most damaging attack, dropping detection by 84.21%, which the authors attribute to disruption of the local bidirectional lexical context that LR-DWM exploits.
• Consistency across models. On DREAM-7B a similar quality-detectability trade-off is observed, with a sharp detectability transition at lower perplexity values, reported in the appendix.

Methodology in Plain English

The scheme treats watermarking as a set of local constraints on the finished text rather than as a property of a generation order.

• Two hash keys, two green lists. For a token at position i, the left neighbor y_i−1 is hashed with secret key k_L, and the right neighbor y_i+1 is hashed with secret key k_R. Each hash partitions the vocabulary into a preferred ("green") set and its complement. If a neighbor is unavailable, that green set is empty.
• Additive logit bias. During denoising, if a neighbor is already revealed, a fixed bias δ is added to the logits of the tokens in the corresponding green set. Because the two constraints are applied independently, their contributions add, so tokens satisfying both constraints are favored most.
• A ternary detection score. Each generated token is compared against both green lists. A match on both neighbors scores +1, a match on one scores 0, and no match scores −1. Under the assumption of human-written text the expected score is 0 and the variance is 1/2 under the random-hash null.
• Standardized test statistic. The scores are summed over the sequence and divided by σ√T, where σ² is estimated empirically from human-written text to account for weak local dependence between adjacent scores. The resulting Z-score is compared against a threshold calibrated for a chosen false positive rate.
• Evaluation protocol. Two diffusion models were tested: LLaDA-8B-Instruct with deterministic decoding, greedy refinement, and a block length of 25, and DREAM-7B-Instruct with its standard stochastic decoding. Both generated 300-token sequences over 300 diffusion steps, using the 600 prompts from WaterBench with the post-generation filtering protocol of Gloaguen et al. (2025). Detectability was measured as true positive rate at 1% false positive rate, text quality as perplexity using Qwen2.5-32B as an external oracle, and efficiency as wall-clock time and peak GPU memory on a single NVIDIA H100.
• DREAM configuration. The DREAM experiments used GEN_LENGTH = 300, STEPS = 300, TEMPERATURE = 0.2, entropy-based stochastic decoding (ALG = entropy, ALG_TEMP = 0.0), EPS = 1e-3, TOP_P = 0.95, and no top-k truncation; a substantial fraction of generated texts failed the quality and length filters, so detection results are computed over the surviving subset. WM-DLM did not produce stable generations under this configuration, so its results use the setup recommended in the original WM-DLM work, with identical evaluation metrics and filtering.

Why This Matters

Watermarking is a mechanism for detecting and attributing AI-generated content. As diffusion language models become a low-latency alternative to autoregressive models, watermarking methods built for sequential decoding become invalid, and the existing adaptations carry significant computational or memory overhead. This paper shows that a watermark can be embedded in DLMs without giving up efficiency, which matters both for research on DLM decoding and for practical deployment.

Real-world applications:

• Content provenance and attribution. Flagging text as machine-generated to support editorial and platform-level labeling.
• Platform moderation and abuse detection. Identifying synthetic text in bulk content streams where attribution matters.
• Academic and publishing integrity. Screening submitted text for AI generation.
• Model auditing and deployment monitoring. Tracking whether a specific model's outputs appear in downstream use.

Industry relevance: The efficiency results are directly relevant to serving costs, since DMARK nearly doubles peak GPU memory and WM-DLM adds runtime overhead, whereas LR-DWM is reported as remaining close to the non-watermarked baseline. For providers running diffusion language models at scale, a watermark with negligible memory and latency cost is far easier to enable by default.

Future Directions

• Adversarial robustness. The paper evaluates only standard non-adaptive perturbations, not adversarial attacks, leaving that threat model open.
• Paraphrase resistance. Paraphrasing reduced detection to 15.79%, so methods that survive paraphrase-level rewriting of the local lexical context remain an open problem.
• The high-detection-rate quality cost. The authors note that while the perplexity increase is small, the decrease in quality can be non-negligible when very high detection rates are required.
• Minimum text length and cross-model generality. Watermarking requires a minimal text length for statistical significance, and the reported gap between LR-DWM and DMARK is larger on DREAM than on LLaDA, which the authors attribute to decoding dynamics and filtering under low-temperature diffusion sampling rather than to the watermark signal itself.

Target Audience

Researchers working on language model watermarking, trustworthy or responsible AI, and diffusion-based text generation will find the core contribution relevant, as will practitioners deploying diffusion language models who need detection capability without a memory or latency penalty. Readers with a background in autoregressive watermarking will be able to follow the adaptation to non-sequential decoding with moderate effort; readers without exposure to logit biasing, hashing-based green lists, or diffusion denoising will need background reading first.

Authors’ abstract

Watermarking (WM) is a critical mechanism for detecting and attributing AI-generated content. Current WM methods for Large Language Models (LLMs) are predominantly tailored for autoregressive (AR) models: They rely on tokens being generated sequentially, and embed stable signals within the generated sequence based on the previously sampled text. Diffusion Language Models (DLMs) generate text via non-sequential iterative denoising, which requires significant modification to use WM methods designed for AR models. Recent work proposed to watermark DLMs by inverting the process when needed, but suffers significant computational or memory overhead. We introduce Left-Right Diffusion Watermarking (LR-DWM), a scheme that biases the generated token based on both left and right neighbors, when they are available. LR-DWM incurs minimal runtime and memory overhead, remaining close to the non-watermarked baseline DLM while enabling reliable statistical detection under standard evaluation settings. Our results demonstrate that DLMs can be watermarked efficiently, achieving high detectability with negligible computational and memory overhead.