Gap-K%: Measuring Top-1 Prediction Gap for Detecting Pretraining Data

AI summary

Overview

• Research area: LLM privacy and security, specifically pretraining data detection (also called membership inference), with connections to data contamination, copyright, and personally identifiable information (PII) exposure.
• Technical level: Intermediate. The paper assumes familiarity with autoregressive language models, token log probabilities, cross-entropy gradients, and AUROC evaluation, but the core intuition is explained from first principles.
• Scope: The paper proposes Gap-K%, a reference-free, gray-box detection method that scores a text by the normalized gap between the model's top-1 predicted token and the actual target token, smoothed over a sliding window, and evaluates it on WikiMIA and MIMIR.

What This Paper Is About

Large language models are pretrained on enormous web-crawled corpora that are usually not disclosed, which creates privacy, copyright, and benchmark-contamination risks. Pretraining data detection tries to answer a simple question: was this specific text in the model's training data or not? The authors argue that existing state-of-the-art detectors (Min-K% and Min-K%++) rely only on how likely each token is and ignore two things: whether the model's single most confident guess matched the actual token, and how membership signals span several adjacent tokens.

Key Contributions

1. Identifying the top-1 prediction gap as a detection signal. The authors analyze the gradient of the next-token prediction loss, ∂ℓt/∂st(v) = p(v | x<t) − 1[v = yt], and observe that the non-target token with the highest probability exerts the strongest gradient unless it equals the target. They conclude that training explicitly penalizes cases where the model's top-1 prediction differs from the target token, so large gaps should be more common in non-training data.
2. Proposing Gap-K%. A detection score built from the normalized difference between the target token's log probability and the maximum (top-1) log probability over the vocabulary, divided by the standard deviation of the next-token log probabilities.
3. Adding sequential smoothing. A sliding window of size w averages the per-token gap scores before selecting the bottom k % of them, so membership is captured over contiguous spans rather than isolated tokens.
4. Demonstrating state-of-the-art results on WikiMIA and MIMIR across multiple model families and sizes, plus generalization tests on recent models and an adversarial paraphrasing stress test. Code is released at https://github.com/meaoww/gap-k.

Main Findings

• WikiMIA AUROC (original setting): Averaged across five models, Gap-K% improves absolute AUROC by 9.7% over the average of the existing baselines and by 2.4% over the strongest baseline, Min-K%++. Averaged over models for the original setting, the improvement over Min-K%++ is 2.1%, 2.6%, and 2.6% at input lengths 32, 64, and 128 respectively.
• WikiMIA AUROC (paraphrased setting): Absolute gains of 5.7% over the baseline average and 1.7% over Min-K%++ when averaged across models. Per length, the gains over Min-K%++ are 0.9%, 2.3%, and 2.0%.
• Average AUROC on WikiMIA (Table 1, averaged over five models): At length 32, Gap-K% reaches 77.8 (original) and 74.3 (paraphrased) versus 75.7 and 73.4 for Min-K%++; at length 64 it reaches 78.4 and 71.2 versus 75.8 and 68.9; at length 128 it reaches 77.4 and 70.6 versus 74.8 and 68.6.
• TPR@5%FPR on WikiMIA: Gap-K% consistently beats Min-K%++ across input lengths, with average gains of 7.1%, 7.9%, and 3.0% in the original setting and 2.0%, 3.4%, and 2.1% in the paraphrased setting for lengths 32, 64, and 128. The paper notes this metric is inherently high-variance because it depends on the extreme tail of the score distribution.
• MIMIR results: MIMIR is described as more challenging because members and non-members come from nearly identical distributions; most methods sit near random guessing (0.5), and the Pile-CC subset is especially hard. Gap-K% nonetheless attains the highest average performance across Pythia models from 1.4B to 12B parameters, exceeding Min-K%++ at every evaluated model size (average AUROC 52.1/53.9/54.6/56.1/57.3 for 160M, 1.4B, 2.8B, 6.9B, 12B versus 52.0/53.7/54.5/55.9/57.1).
• Empirical support for the gap signal: On WikiMIA-64 with Pythia-12B, non-training data contains a higher fraction of large-gap tokens at every threshold tested. At τ = 3, deviations of more than three standard deviations from the top-1 prediction occur in 39.9% of non-training tokens versus 35.5% of training tokens. At τ = 5, the figures are 20.37% versus 16.38%.
• Sequential locality matters: Sequential smoothing on the original token order gives 74.8 AUROC, versus 72.3 with no smoothing and 72.9 with smoothing after shuffling the token order.
• Ablation of components: Starting from Min-K%++ at 72.6, replacing the mean with the Top-1 gap alone gives 72.3, applying smoothing alone gives 73.8, and combining both (Gap-K%) gives 74.8. The authors conclude the Top-1 gap only becomes effective when combined with smoothing.
• Effect of k: Performance peaks around k = 15%, and Gap-K% outperforms Min-K% and Min-K%++ across the entire 5% to 50% range tested.
• Window size: LLaMA-13B performs best at window size 6 and Pythia-12B at window size 3, with degradation at larger windows; the paper attributes this to smoothing diluting localized high-gap regions.
• Generalization to recent models: On WikiMIA-25 (length 64 split), Gap-K% leads all baselines on LLaMA 3.1-8B (84.1 versus 82.7 for Min-K%++), LLaMA 3.1-8B Instruct (76.6 versus 73.1), Gemma2-9B (78.3 versus 75.6), and Gemma2-9B Instruct (65.8 versus 64.5).
• Robustness to adversarial paraphrasing: Using DIPPER with lexical diversity L = 60 and order diversity O = 60, Gap-K% reaches 66.6 AUROC, compared with 65.5 for Min-K%++, 60.3 for Neighbor, 57.9 for Min-K%, 52.3 for Loss, and 50.0 for Zlib.
• Relationship to Min-K%++: The score can be written as g_t = z_t − Δ_t, where Δ_t is the normalized gap between the top-1 log probability and the mean. Min-K%++ measures deviation from the mean of the vocabulary distribution, while Gap-K% measures the gap to the mode, letting it penalize confident mispredictions rather than treating all low-probability tokens alike.

Methodology in Plain English

The starting point is a property of how language models are trained. At each position, the model is penalized according to how much probability it assigned to the correct next token, and the gradient pushes hardest against whichever wrong token the model liked best. Over training, this pressure teaches the model to make its single top guess match the actual next token for text it has seen. For text it has not seen, the model falls back on general language patterns, and sometimes it confidently predicts a plausible token that is not the one that actually appeared. That mismatch is the signal this paper exploits.

To turn this into a measurable score, the authors take the difference between the log probability of the actual token and the log probability of the model's top-1 choice, then divide by the standard deviation of the log probabilities over the vocabulary so that peaked and flat distributions are comparable. The resulting score is always zero or negative: near zero means the model's best guess was essentially the real token, while a large negative value means the model confidently preferred something else.

Because a single token is noisy, the authors average these scores over a sliding window of neighboring tokens before doing anything else. Finally, following the logic of Min-K%, they ignore the easy parts of the text and take the average of the bottom k % of the smoothed scores, on the reasoning that a sample where even its worst moments show only small gaps is likely to be a training sample.

Evaluation uses AUROC and TPR@5%FPR on WikiMIA (with original and paraphrased versions, and length splits of 32, 64, and 128) and on MIMIR (seven domains from the Pile, evaluated on Pythia-160M through 12B). The setting is gray-box: the method needs token-level probabilities but not the model's parameters or gradients. Comparisons are made against Loss, Zlib, Neighbor, Min-K%, and Min-K%++, with k fixed at 20% for the three K%-style methods to keep the comparison fair, and window size set to 6 for LLaMA-based models and 3 for other models.

Why This Matters

• Impact on research: The paper argues that the field has been fixated on token likelihoods and has overlooked the ranking behavior of the model, and that membership signals are sequential rather than isolated. Framing detection through the optimization dynamics of next-token prediction gives a mechanistic reason to expect certain statistics to differ between members and non-members, rather than relying on empirical heuristics.
• Real-world applications:
  • Auditing whether private or personally identifiable information was absorbed into a model's pretraining corpus.
  • Detecting benchmark contamination, where test sets leaked into training data would inflate reported performance and make comparisons unfair.
  • Investigating copyright exposure for specific published texts.
  • Documenting memorization behavior for model cards, compliance reviews, or regulatory reporting.
• Industry relevance: Model providers and API vendors increasingly face questions about what data a model saw. A reference-free method is attractive because it needs only one deployed model and no costly reference-model training runs, which makes it practical for large proprietary systems. The gray-box requirement for token-level probabilities is the main constraint, since most public APIs expose only sampled text.

Future Directions

• Extending to fully black-box settings. The method requires access to token-level probability outputs. The authors identify this as a limitation since some APIs expose only generated text, and call generalization to black-box access an important direction.
• Testing larger and more diverse models. Evaluation covers models up to tens of billions of parameters and includes LLaMA 3.1 and Gemma2 with their instruction-tuned variants, but the authors state that behavior at the hundreds-of-billions scale is unverified and that many other recent model families are not covered.
• Defending against detection-aware attacks. The DIPPER evaluation uses strong paraphrasing, but the authors caution that it does not capture adaptive adversaries who know the detection mechanism and deliberately manipulate token-level statistics.
• Understanding why the optimal window size differs by model family. LLaMA-13B and Pythia-12B have different optimal window sizes (6 and 3), and the paper defers an investigation of this difference to an appendix rather than resolving it in the main text.

Target Audience

Researchers and practitioners working on LLM privacy, memorization, and data provenance who already understand token-level probabilities and ROC-based evaluation. It is also relevant to benchmark maintainers worried about contamination, to legal and compliance teams assessing training-data exposure, and to anyone building or auditing data-detection tooling, since the method is lightweight and reference-free but requires gray-box access to model output probabilities.

Authors’ abstract

The opacity of massive pretraining corpora in Large Language Models (LLMs) raises significant privacy and copyright concerns, making pretraining data detection a critical challenge. Existing state-of-the-art methods typically rely on token likelihoods, yet they often overlook the gap between the target token and the model's top-1 prediction, as well as local correlations between adjacent tokens. In this work, we propose Gap-K%, a novel pretraining data detection method grounded in the optimization dynamics of LLM pretraining. By analyzing the next-token prediction objective, we observe that discrepancies between the model's top-1 prediction and the target token induce strong gradient signals, which are explicitly penalized during training. Motivated by this, Gap-K% leverages the log probability gap between the top-1 predicted token and the target token, incorporating a sliding window strategy to capture local correlations and mitigate token-level fluctuations. Extensive experiments on the WikiMIA and MIMIR benchmarks demonstrate that Gap-K% achieves state-of-the-art performance, consistently outperforming prior baselines across various model sizes and input lengths.