How Sampling Affects the Detectability of Machine-written texts: A Comprehensive Study

AI summary

Overview

• Research area: Natural Language Processing, specifically Artificial Text Detection (ATD) and the generation dynamics of Large Language Models.
• Technical level: Intermediate. The headline findings are intuitive, but the paper includes formal definitions of sampling adapters, lexical diversity metrics, and probability divergence measures.
• Scope: A systematic study of how 37 sampling-based decoding configurations change the detectability of machine-written text, evaluated with one supervised detector (RoBERTa) and two unsupervised detectors (Binoculars and FastDetectGPT).

What This Paper Is About

Detectors of machine-written text often report near-perfect results, with AUROC scores above 99%, but those results are usually tied to one fixed generation setting chosen by the detector's authors. This paper asks how stable those claims are when the text generator's decoding strategy changes: the temperature, the truncation threshold, the repetition penalty, and so on. The goal is to map out exactly which sampling choices break detection, why they break it, and what that implies for how detectors are evaluated.

Key Contributions

1. A large-scale benchmark dataset with texts generated using six decoding strategies across 37 decoding configurations, enabling fine-grained analysis of detector behaviour.
2. Extensive empirical evaluation showing that state-of-the-art detectors are highly sensitive to generation parameters, with drastic performance drops (for example AUROC falling from 0.99 to 0.01).
3. An in-depth analysis of the failure mechanisms, identifying which distributional properties predict detection success and failure, and how generation dynamics interact with detectability.
4. A public release of the dataset, code, and evaluation framework at https://github.com/BaggerOfWords/Sampling-and-Detection.

Main Findings

• Modest parameter changes destroy detection. Even minor adjustments to decoding parameters such as temperature, top-p, or nucleus sampling severely impair detector accuracy. The paper reports AUROC falling from near-perfect levels to 1% in some settings, and the conclusion describes drops "from near-perfect down to null" achieved simply by tuning the repetition penalty.

• Repetition penalty is the most damaging setting. With Binoculars, AUROC is 0.0711 at repetition penalty 1.05 and falls to 0.0157-0.0182 across values 1.10 to 1.30. With FastDetectGPT, AUROC sits between 0.3235 and 0.5699 across the same range, meaning the two score distributions are essentially blended together.

• High temperature is the second major failure mode. Binoculars reaches 0.9928 at temperature 0.5 and 0.9449 at 0.9, then collapses to 0.0200 at 1.1, 0.0056 at 1.2, and 0.0019 at 1.3. FastDetectGPT degrades more gradually, from 0.9468 at 0.5 to 0.3693 at 1.1 and 0.1672 at 1.3. The authors describe the high-temperature outputs as becoming "too human-like", i.e. surprising, for Binoculars.

• Truncation-based samplers are comparatively safe. Binoculars peaks at 0.9975 for top-p 0.7, reaches 0.9916 at top-k 10, and 0.9933 at typical sampling 0.8. Performance is weakest at the extremes of each family: top-k 1000 gives Binoculars 0.6355, top-p 0.95 gives 0.8625, typical 0.95 gives 0.8703.

• η-sampling behaves differently at each end. Binoculars drops to 0.7672 at η = 1e-4 but rises to 0.9965 at η = 0.05, even though the small η value is the one whose lexical diversity most closely matches human text.

• Supervised detectors generalise poorly across samplers. A RoBERTa-base classifier trained on one decoding parameter and tested on another fails noticeably when training used repetition penalty or high temperatures. Training on a uniformly sampled mixture of the training data solves the misclassification issues almost entirely, yielding an accuracy of at least 95% for all tested parameters. Training on a mixture of parameters close to human diversity levels gives an accuracy of 94% averaged on all parameters, dragged down by 87% at T=1.2 and 80% at T=1.3.

• Mixture training still leaves a brittle detector. When the human subset of the RAID test data (BBC News articles) was replaced with news articles from the CCNews subset of Common Crawl, detection accuracy dropped to 72% on average. The authors say this suggests the classifier is mostly overfitting its human training dataset.

• Uniform mixture detector models help only slightly. Replacing the main detector model with a uniform mixture of all 37 decoding-induced distributions improved performance for both Binoculars and FastDetectGPT in all settings, except the two troublesome ones (repetition penalty and temperature above 1).

• Entropy and model-to-model divergence predict failure, not perplexity. The strongest correlations with AUROC are the averaged entropy of the main model over the text (-0.910 for Binoculars, -0.935 for FastDetectGPT) and the divergence between the two detector models (KL -0.850 and -0.898; Rényi α=0.2 -0.890 and -0.921; Rényi α=1.2 -0.828 and -0.882). Perplexity is a weaker predictor (-0.458 and -0.611), and the divergence between the original generator distribution and its adapted version is weaker still (TV -0.056 and +0.050; Euclidean -0.254 and -0.118). Cross-entropy was the only positively correlated indicator (+0.502 and +0.526).

• Human-like diversity settings are identifiable. The configurations whose lexical diversity metrics come closest to human text are η = 10^-4, T = 1.0, k = 100, p = 0.95, and τ = 0.95. Human samples in the dataset score MTLD 94.60, Hapax 34.90, Simpson 0.66, Zipf α 1.20, Heaps 59.46, average length 389.38, and perplexity 14.32, while high-temperature settings overshoot dramatically (MTLD 7081.06 at T=1.2 and 15264.92 at T=1.3).

• Extreme diversity degrades quality in two opposite ways. Over-diverse sampling pulls tokens from very low-probability regions, producing nonsensical output with grammatical errors and even language changes. Under-diverse sampling concentrates mass on frequent types, producing repetitive n-grams that feel artificial.

Methodology in Plain English

The researchers started from 2,000 English human-written texts drawn from the RAID dataset, covering 11 different domains, together with the prompts used to produce them. Using Llama-3.2-3B, they generated 37 machine-written counterparts for each human text: 36 using the sampling adapters with specific hyperparameter values, plus one using plain ancestral sampling. Generation used Hugging Face's .generate() function with max_tokens set to 512 to approximately match the length of the human samples. The data was split 1:1 into training and test sets.

They then measured each generated dataset's lexical diversity using five metrics (MTLD with a threshold of 0.72, hapax legomena ratio, Simpson's diversity index, Zipfian exponent, and Heaps' law exponent), plus average length and perplexity, and used the human samples as a gold-standard profile.

For detection, they tested three methods in what they call a "best-case scenario": a fine-tuned RoBERTa-base classifier (learning rate 5e-05 for 3 epochs with AdamW, the default TrainingArguments), and two unsupervised methods, Binoculars and FastDetectGPT, using Llama-3.2-3B and Llama-3.2-3B-Instruct as the underlying model pair. All supervised experiments trained and tested on the same generator model. They also tried a variant where the main detector model was replaced by a uniform mixture of all 37 decoding-induced distributions.

Finally, to explain the failures, they correlated detector AUROC against two families of indicators: divergences between the adapted generator distribution and the main detector model, and divergences between the two detector models, using metrics including total variation distance, cross-entropy, KL divergence, Rényi divergence, and Euclidean distance.

Why This Matters

Impact on research. The paper argues that current detection benchmarks do not sufficiently represent all decoding strategies, and that reported near-perfect accuracy figures may not transfer to other use cases. It exposes a specific blind spot: optimal detection scores depend on the two detector models assigning very similar probabilities, so the failure of unsupervised methods is structural, not incidental.

Real-world applications.

• Content moderation and platform trust-and-safety pipelines that rely on machine-text detectors to flag automated or deceptive content.
• Academic integrity and publishing workflows, where detectors are used to screen submissions and student work.
• Journalism and fact-checking, where verification teams need to know whether a text came from a model or a person.
• Legal, forensic, and provenance settings, where claims about a text's origin carry consequences.

Industry relevance. Detector vendors, LLM providers, and evaluators building benchmarks all depend on detection robustness claims. The paper's finding that changing a single generation parameter can push AUROC from near-perfect to near-chance means that any deployed detector evaluated under one decoding configuration may fail silently when a user changes settings. The released dataset, code, and evaluation framework are intended to make this kind of testing routine.

Future Directions

1. Test more generator models and architectures. The study uses Llama-3.2-3B as the only generator, so it is not reported whether the same failure patterns hold for larger or differently trained models.
2. Cover more elaborate decoding techniques. The authors note that if changing one parameter seriously derails detectors, more elaborate methods such as Minimum Bayes Risk (MBR) decoding or Monte Carlo Tree Search (MCTS) would likely do so as well, and should be evaluated.
3. Build detectors robust to sampling. New detection methods are needed that do not degrade when the (sub)word-level distribution shifts, since current supervised and unsupervised approaches both fail.
4. Improve evaluation protocols and address domain overfitting. The drop to 72% accuracy under a news-source swap shows that supervised detectors overfit their human training data, suggesting benchmark design and cross-domain generalisation as open problems.

Target Audience

Researchers and practitioners working on machine-generated text detection, LLM evaluation, and benchmark design. It is also relevant to applied teams deploying detectors in production who need to understand the conditions under which accuracy claims hold, and to anyone studying how decoding strategies shape the statistical properties of generated text. Readers without a background in probability theory can follow the experimental results and the key takeaways, though the sampling-adapter and divergence-metric sections assume familiarity with distributional notation.

Authors’ abstract

As texts generated by Large Language Models (LLMs) are ever more common and often indistinguishable from human-written content, research on automatic text detection has attracted growing attention. Many recent detectors report near-perfect accuracy, often boasting AUROC scores above 99\%. However, these claims typically assume fixed generation settings, leaving open the question of how robust such systems are to changes in decoding strategies. In this work, we systematically examine how sampling-based decoding impacts detectability, with a focus on how subtle variations in a model's (sub)word-level distribution affect detection performance. We find that even minor adjustments to decoding parameters - such as temperature, top-p, or nucleus sampling - can severely impair detector accuracy, with AUROC dropping from near-perfect levels to 1\% in some settings. Our findings expose critical blind spots in current detection methods and emphasize the need for more comprehensive evaluation protocols. To facilitate future research, we release a large-scale dataset encompassing 37 decoding configurations, along with our code and evaluation framework https://github.com/BaggerOfWords/Sampling-and-Detection