Distilling the Thought, Watermarking the Answer: A Principle Semantic Guided Watermark for Large Reasoning Models

AI summary

Overview

Research area: AI security and provenance — specifically digital watermarking for Large Language Models (LLMs), applied to reasoning-intensive LLMs (RLLMs) that produce chain-of-thought "thinking" traces before a final answer.

Technical level: Advanced. The paper builds on prior token-based watermarking (KGW-style green/red list partitioning) and defines its method through information-theoretic quantities (causal divergence, competitive entropy reduction, JS divergence) and a PCA-derived vector. Readers need working familiarity with transformer decoding, logits, perplexity, and watermark detection via statistical z-tests.

Scope: The paper proposes ReasonMark, a two-phase watermarking framework that leaves the model's reasoning trace unmodified and embeds a semantically guided watermark only in the final answer, evaluated on text completion, machine translation, and mathematical reasoning benchmarks with Qwen3-32B and DeepSeek-R1-Distill-Qwen-32B.

What This Paper Is About

Watermarking lets a model owner invisibly embed a detectable signal in generated text so outputs can be traced. Existing methods were built for general-purpose LLMs: token-based approaches apply pseudo-random biases that can corrupt the logical flow of a reasoning model's chain of thought, while semantic-aware approaches preserve quality but add latency or require auxiliary models. ReasonMark's goal is to embed a detectable watermark in a reasoning LLM's output while leaving its internal reasoning untouched and its final answer logically coherent.

Key Contributions

1. A two-phase watermarking framework for reasoning models. It decouples generation into an undisturbed Thinking Phase (the internal chain-of-thought, left completely unmodified) and a watermarked Answering Phase (the final user-facing response). The authors state this is the first approach specifically designed to protect RLLM outputs without corrupting logical integrity.

2. A principled method for distilling the reasoning process into a semantic guide. A Criticality Score selects semantically pivotal tokens ("Critical Tokens," CTs) from the reasoning trace, and these are transformed via PCA into a continuous Principal Semantic Vector (PSV) that provides a directional guide for watermarking.

3. A semantically adaptive embedding mechanism. Watermark strength is modulated per token according to that token's alignment with the current PSV, rather than applying a single fixed bonus to all green-list tokens.

4. Reported gains over state-of-the-art baselines. The paper reports reduced text perplexity (−0.35), higher translation BLEU (+0.164), higher mathematical accuracy (+0.67 points), and a 0.34% higher watermark detection AUC, with a negligible increase in latency.

Main Findings

• Text quality (C4): ReasonMark achieves the lowest perplexity among all watermarking methods — 10.31 for Qwen3-32B and 10.54 for DeepSeek-R1-Distill-Qwen-32B. The non-watermarked baselines are 10.55 and 10.82 respectively, so on Qwen3 the watermarked output is slightly lower perplexity than unwatermarked text. The closest competing method on Qwen3 is Unigram at 10.66, giving the reported reduction of 0.35.

• Machine translation (WMT16-DE-EN): ReasonMark obtains the highest BLEU among watermarking techniques — 9.916 (Qwen3-32B) and 9.653 (DeepSeek). Non-watermarked baselines are 7.851 and 7.622. The reported +0.164 improvement corresponds to the gap against the best baseline on Qwen3, MorphMark at 9.752.

• Mathematical reasoning (AIME and GSM8K): ReasonMark reports the highest mathematical accuracy in each setting — AIME 69.86 (Qwen3) and 71.34 (DeepSeek); GSM8K 93.96 (Qwen3) and 95.14 (DeepSeek). On DeepSeek AIME, 71.34 approaches the non-watermarked baseline of 71.52. In every setting the non-watermarked baseline is numerically slightly higher than ReasonMark (AIME Qwen3 70.03; GSM8K Qwen3 94.01; GSM8K DeepSeek 95.21), and the paper describes its results as "closely matching or even slightly exceeding the baseline."

• Detection: ReasonMark reports the highest or near-highest detection AUC throughout — 99.31 and 99.52 on C4, and above 99.9 on the AIME reasoning task (table values 99.95 for Qwen3 and 99.98 for DeepSeek). This is the basis for the claimed 0.34% AUC improvement over the strongest baseline.

• Attack robustness: Evaluated on C4 with Qwen3 under two attack types. Attack A1 applies random word-level perturbations (insertions, deletions, synonym substitutions) with 30 tokens modified. Attack A2 applies semantic-level transformations via translation and paraphrasing through the DeepSeek-V3 API (translation into Chinese and back to English). From an unattacked AUC of 99.31, ReasonMark stays above 93.5 under word deletion, insertion, and synonym replacement, and retains AUC of 82.58 against translation and 70.54 against paraphrasing.

• Ablation (C4, Qwen3): Full ReasonMark reaches PPL 10.3080 and AUC 0.9931. Replacing principled CT selection with random sampling (w/o CTs) causes the largest degradation, PPL rising to 12.8801. Removing Global Causal Contribution (w/o GCC) raises PPL to 11.1510 while AUC stays at 99.11; removing Competitive Persistence Scoring (w/o CPS) yields PPL 11.0597 and drops AUC to 98.69. The paper concludes GCC drives semantic coherence while CPS drives detection robustness.

• Hyperparameter behavior: For the PSV update rate β₀, perplexity follows a U-shape — 11.1 at 10⁻³, a minimum near 10.3 around 0.1, then rising again. For top-k, perplexity is highest at k=3, lowest around k=10, and gradually rises toward k=100. The optimal configuration (β₀ between 0.01 and 0.1, top-k between 10 and 50) produces perplexity better than non-watermarked text. In the δ₀/δλ interaction study, δ₀ affects AUC more strongly while δλ affects PPL more strongly.

• Detection procedure unchanged: Detection remains stateless and requires no access to the PSV or the original prompt; it uses the same statistical z-test as KGW, checking for a statistically significant bias toward green-list tokens.

• Not reported in the provided content: The specific latency measurements (stated to be in Appendix F.2), the full translation metric breakdown (Appendix F.1), dataset sample counts, and the detailed attack robustness tables (Appendix F.3) are referenced but their numbers do not appear in the truncated text provided.

Methodology in Plain English

The core idea is to separate what the model thinks from what it says, and only watermark the latter.

1. Split the generation. The full output is divided at a structural delimiter (for example <think> markers), giving a Thinking Phase of N tokens and an Answering Phase of the remaining tokens. The thinking phase is generated with no watermark at all, so reasoning quality is untouched.

2. Score the reasoning tokens. For every word in the vocabulary, the method computes a Criticality Score. This combines two signals. The first, Global Causal Contribution, rewards words that keep appearing with high probability across causally connected steps of the reasoning, weighted so that moments of large change in the model's predictive distribution (measured by JS divergence between consecutive steps) count more. The second, Competitive Persistence Scoring, rewards words that win or come close to winning in tightly contested next-token choices, weighted so that high-confidence choices count more, and counts how often the word stays in the top-k candidates over the following steps.

3. Turn the top tokens into a direction. The K highest-scoring tokens are collected, their embeddings stacked into a matrix, and PCA is applied. The first principal component becomes the initial Principal Semantic Vector (PSV) — an arrow pointing along the dominant semantic direction of the reasoning.

4. Watermark the answer with that direction. During the answering phase, the vocabulary is split into green and red lists by the standard hash of the previous token. Instead of adding a fixed bonus to every green token, each green token gets a bonus scaled by the cosine similarity between its embedding and the current PSV. Tokens that fit the established reasoning get a stronger push; mismatched ones get less.

5. Let the vector drift. After each generated token, the PSV is updated via an exponential moving average, with the update rate scaled by how semantically relevant the new token is. The watermark therefore tracks the evolving answer while staying anchored to the original reasoning.

6. Detect normally. Verification just checks whether green-list tokens appear more often than chance, using the standard KGW z-test — no extra model, no stored state.

Why This Matters

Impact on research. The paper reframes the watermarking trade-off. Instead of accepting that a stronger watermark degrades text, it argues the watermark should follow the model's own reasoning direction, so strength and coherence reinforce each other rather than conflict. Since detection requires no change to the existing KGW procedure, the method is compatible with established verification pipelines. The ablation result that random token selection (w/o CTs) pushes perplexity to 12.8801 also gives direct evidence that which tokens guide the watermark matters, not just that guidance exists.

Real-world applications:

• Provenance and attribution for AI-generated math and code. Because reasoning models are deployed for problem-solving, a watermark that survives without corrupting the chain of thought supports tracing which model produced a solution.
• Machine translation pipelines. With BLEU reported as the highest among watermarking methods, the approach targets commercial translation services that need output watermarking without fidelity loss.
• IP protection for model providers. Providers can embed invisible ownership signals in served outputs while preserving the response quality customers pay for.
• Adversarial content forensics. Robustness above 93.5 AUC under word-level edits and 82.58 under translation attacks means the signal can persist after a user has lightly rewritten or round-tripped text through another language.

Industry relevance. The paper emphasizes near-zero added latency and no auxiliary model, in contrast to semantic-aware methods that require architectural changes or extra inference. For high-throughput serving, avoiding a second model or multiple generation passes is a direct cost consideration, and unchanged detection means existing monitoring infrastructure can be reused.

Future Directions

• Closing the small accuracy gap on reasoning benchmarks. In every reported mathematical setting, the non-watermarked baseline is numerically slightly higher than ReasonMark (for example, AIME Qwen3 at 70.03 versus 69.86). Understanding and eliminating that residual gap is a natural next step.
• Generalizing across reasoning architectures and formats. The phase split relies on a Marker-Based Separation algorithm that detects structural delimiters. How well this transfers to models with different or implicit reasoning formats is an open question.
• Scaling to long reasoning traces and larger critical-token budgets. The sensitivity study shows performance depends on the top-k choice, with perplexity lowest around k=10 and rising toward k=100; behavior on much longer traces and different K values is unresolved.
• Stronger and adaptive attacks. The paper reports AUC falling to 70.54 under paraphrasing. Hardening against semantic rewriting, and testing against adversaries aware of the PSV mechanism, remains work to be done.
• Beyond binary detection. The related-work discussion notes prior multi-bit message encoding via probability-balanced partitioning and error-correcting codes; extending ReasonMark's semantically guided strength modulation to multi-bit payloads is an evident extension.

Target Audience

Researchers and engineers working on LLM watermarking, AI content provenance, and trustworthy model deployment, particularly those serving reasoning models. It is also relevant to practitioners who need to compare watermarking methods on a shared benchmark set (C4, WMT16-DE-EN, AIME, GSM8K with Qwen3-32B and DeepSeek-R1-Distill-Qwen-32B), since the paper reports a broad baseline table against KGW, UPV, Unigram, Unbiased, SynthID, SWEET, EWD, WatMe, MorphMark, SemStamp, k-SemStamp, and SimMark. The mathematical formulation of the Criticality Score makes the paper Advanced-level; readers wanting only the design intuition can read the Introduction, Figure 1 description, and Section 3.2.

Authors’ abstract

Reasoning Large Language Models (RLLMs) excelling in complex tasks present unique challenges for digital watermarking, as existing methods often disrupt logical coherence or incur high computational costs. Token-based watermarking techniques can corrupt the reasoning flow by applying pseudo-random biases, while semantic-aware approaches improve quality but introduce significant latency or require auxiliary models. This paper introduces ReasonMark, a novel watermarking framework specifically designed for reasoning-intensive LLMs. Our approach decouples generation into an undisturbed Thinking Phase and a watermarked Answering Phase. We propose a Criticality Score to identify semantically pivotal tokens from the reasoning trace, which are distilled into a Principal Semantic Vector (PSV). The PSV then guides a semantically-adaptive mechanism that modulates watermark strength based on token-PSV alignment, ensuring robustness without compromising logical integrity. Extensive experiments show ReasonMark surpasses state-of-the-art methods by reducing text Perplexity by 0.35, increasing translation BLEU score by 0.164, and raising mathematical accuracy by 0.67 points. These advancements are achieved alongside a 0.34% higher watermark detection AUC and stronger robustness to attacks, all with a negligible increase in latency. This work enables the traceable and trustworthy deployment of reasoning LLMs in real-world applications.