Log Probability Tracking of LLM APIs

AI summary

Overview

Research area: Machine learning systems / LLM API auditing and change detection.

Technical level: Intermediate. The core method is a simple two-sample permutation test, but the paper assumes familiarity with logits, log probabilities, tokenization, fine-tuning, LoRA, pruning, and ROC AUC.

Scope: The paper proposes and evaluates "logprob tracking" (LT), a statistical method that monitors the log probabilities of a single output token to detect when an LLM API's served model or infrastructure has changed, and applies it to 189 real-world endpoints over more than 4 months.

What This Paper Is About

When developers, researchers, or regulators use an LLM through an API, they expect the endpoint to keep serving the same model over time, but they have no practical way to verify that. Existing audit methods are expensive because they require many queries, long responses, or extensive benchmarking, so in practice API changes go largely unmonitored. This paper shows that the log probabilities returned alongside tokens, despite being non-deterministic, can be used as the basis for continuous monitoring that is far cheaper and more sensitive than existing approaches.

Key Contributions

1. The logprob tracking (LT) method: A two-sample permutation test on the mean absolute distance between per-token average logprobs, requiring only a 1-token prompt and a 1-token response. The authors report it exceeds the detection performance and sensitivity of alternative methods at a fraction of the cost.

2. The TinyChange benchmark: A publicly released benchmark that takes an LLM as input and generates 58 variants spanning five levels of modification intensity (difficulty increasing in powers of two), covering regular fine-tuning, LoRA fine-tuning, weight pruning (by magnitude or random selection), and Gaussian parameter noising.

3. Extensive in-vitro evaluation: LT is compared against two baselines (MET and MMLU-ALG) across five open-weight models and 290 variants, on ROC AUC, sensitivity, and token cost.

4. A real-world deployment study: Hourly monitoring of 189 API endpoints from 10 providers over more than 4 months, collecting more than 1.7M responses, which surfaced 37 suspected changes.

Main Findings

• Logprobs are widely but not universally available: Among 813 endpoints listed on OpenRouter (a gateway to 60+ LLM API providers and 500+ models), 710 returned a correct response to a single "x" prompt, and 164 of those (23%) complied when logprobs were requested. Fireworks AI and Azure returned 5 logprobs, xAI 8, and other providers 20.

• Short prompts are sufficient: Across models and variants, the shortest prompt (1.5 tokens) performed within about 1% AUC of the longest prompt (33 tokens), against a baseline of around 91% AUC. The authors therefore use the single letter "x" (1 to 2 tokens depending on tokenizer) for the rest of the experiments.

• LT outperforms both baselines overall: Overall ROC AUC across the TinyChange benchmark and models was 0.915 (95% CI 0.864, 0.958) for LT, versus 0.878 (0.802, 0.944) for MMLU-ALG and 0.670 (0.612, 0.731) for MET.

• LT is dramatically cheaper: Token counts per test (input, output) were (28, 20) for LT, (2.9 × 10⁴, 2.0 × 10⁴) for MET, and (2.1 × 10⁵, 9.9 × 10³) for MMLU-ALG. Cost per year of hourly sampling at GPT-4.1 pricing (input $3, output $12 per 1M tokens) was $0.14 for LT, $146 for MET, and $332 for MMLU-ALG. The paper describes LT as up to three orders of magnitude cheaper, or 1/1,000th of the cost.

• LT detects smaller changes: Using an AUC point estimate of 0.9 as the threshold for good detection, in the weight pruning experiment the highest difficulty level still detected was 2⁻¹ for MET, 2⁻⁴ for MMLU-ALG, and 2⁻¹⁰ (or below) for LT. This is a factor of 2⁹ = 512 between LT and MET, and slightly higher than 2⁶ = 64 between LT and MMLU-ALG. The authors tentatively conclude LT is 2–3 orders of magnitude more sensitive to small changes than MET and 1–2 orders of magnitude more sensitive than MMLU-ALG.

• LoRA fine-tuning is hard to detect for all methods: The paper notes LoRA finetuning is challenging for all methods, and explains this via L2 distance between original and finetuned models (measured for Qwen2.5-0.5B-Instruct): at 1 step, LoRA is 4.3 × 10⁻⁹ versus 1.3 × 10⁻⁵ for full fine-tuning; at 16 steps, 9.5 × 10⁻⁸ versus 1.3 × 10⁻⁴; at 256 steps, 1.4 × 10⁻⁵ versus 3.5 × 10⁻³.

• Real-world changes are prevalent: Among 189 tracked endpoints from 10 providers, LT flagged 37 suspected changes across 29 endpoints and 7 providers. Fireworks had the highest rate (9 changes over 4.1 cumulative endpoint-years, 2.2/year), followed by Lambda (6 changes, 1.9/year) and Azure (2 changes, 1.3/year). OpenAI (19 endpoints, 6.9 endpoint-years), Hyperbolic, and Deepseek recorded 0 changes. The overall rate was 0.86 changes per endpoint-year. 34 of the 37 changes affected open-weight models.

• Providers were unresponsive to queries: The authors contacted providers in June and July 2025 about earlier suspected changes. xAI and Fireworks AI did not respond; Lambda and Nebius AI replied that they could not share details, with Nebius AI adding that they perform changes on a regular basis.

• Possible obstruction observed: Between 16:00 and 17:00 UTC on September 24, 2025, OpenAI began requiring at least 16 tokens of output on the GPT-4.1 line of models accessed via OpenRouter (but not on OpenAI's own API), for one-token requests that asked for output logprobs.

Methodology in Plain English

The intuition is that when an LLM generates a token, it produces a vector of log probabilities over its whole vocabulary; requesting logprobs asks the API to return the top-k of these values. These continuous numbers carry much more information than the single sampled token.

The obstacle is that logprobs are not stable: identical requests return slightly different values because requests share batches with other traffic, and because production requests get routed to different GPUs. The authors treat each returned logprob as a sample from a probability distribution rather than a fixed number.

Their procedure is straightforward. Send the same short prompt to two endpoints (or the same endpoint at two different times), request only one token of output, and repeat N times. Since the top-k truncation means different tokens appear in different samples, a token missing from a sample is conservatively imputed with that sample's minimum logprob. For each token observed across both sets of samples, they compute the average logprob on each side, then take the mean absolute difference between these averages as the test statistic. A permutation test over the pooled samples (splitting them into two groups of size N B times) gives a p-value for the null hypothesis that the two distributions are identical; if that p-value falls below the significance level, they conclude the two APIs differ.

For evaluation, they built TinyChange to generate variants at controlled magnitudes. Fine-tuning data came from LMSYS-Chat-1M, restricted to single-turn GPT-4 conversations, chosen because such data is in-distribution for GPT-4-like models and therefore hard to detect. They applied the benchmark to five open-weight models (Qwen 2.5 0.5B, Gemma 3 1B, Phi-3 Mini 4k, Llama 3.1 8B, OLMo 2 7B), producing 290 variants. Sampling used vLLM on 2x H100 80GB GPUs, with 10,000 samples per prompt for each of the 295 models (5 originals plus 290 variants), giving 416M outputs. To create realistic non-determinism for LT, they mixed 16 tested prompts with 48 random prompts from LMSYS-Chat-1M and shuffled them, reading only the first token's logprobs. For each model they ran 1,000 original/original hypothesis tests and 1,000 original/variant tests, each using N = 10 sampled outputs per prompt, then computed ROC curves from the two sets of statistics.

The two baselines were MET, which uses Maximum Mean Discrepancy with a Hamming distance kernel over 25 prompts and 50-token responses at temperature 1, and MMLU-ALG, which uses the "abstract_algebra" subset of MMLU (100 of the benchmark's 14,042 questions) with multiple-choice answers sampled at temperature 0.1 for 5 output tokens each.

For the live deployment, LT compares adjacent 24-sample windows hourly, computes the running mean and standard deviation of the test statistics over a window of 100 samples, and flags a change when the statistic exceeds the running mean by at least 12 standard deviations and is above 1.0.

Why This Matters

Impact on research: The paper argues that undisclosed model changes threaten the reproducibility of research and the reliability of downstream applications, and that existing audits are too costly to run at regular intervals. By reducing the cost of a hypothesis test to roughly 48 tokens and $0.14 per year of hourly sampling, it makes continuous monitoring of a broad set of APIs practical rather than theoretical. The paper also frames LT as establishing a practical upper bound on detection sensitivity under inference non-determinism, showing that a single fine-tuning step can be reliably detected by a black-box method.

Real-world applications:

• Continuous integrity monitoring of production LLM endpoints by developers who need to catch regressions or silent model swaps.
• Reproducibility checks in research pipelines that depend on a specific model version remaining stable over months.
• Regulatory and compliance audits that need an ongoing signal after an initial assessment, not just a one-time check.
• Internal use by providers themselves, who could integrate LT into their own test suites to monitor production inference.

Industry relevance: The paper reports 37 suspected changes across 29 endpoints and 7 providers, and notes that 34 of the 37 affected open-weight models, where users might reasonably expect greater stability. It also documents an instance where a provider imposed a minimum output length that blocked one-token logprob requests, and discusses incentives for evasion (identifying monitoring queries, or caching logprobs). The authors cite the three 2025 incidents where the version of Grok deployed on X received a modified system prompt, two blamed on rogue employees and one on a bad update, as evidence that these concerns are not purely theoretical.

Future Directions

• Tune the detection method further: The authors state that performance can likely be increased by parameter tuning, and suggest alternatives such as CUSUM control charts instead of the current permutation test framework.

• Extend beyond binary detection: The paper notes the framework naturally extends to quantifying the magnitude of a change, rather than only flagging that one occurred.

• Address the logprob availability constraint: Since LT only applies to APIs that return logprobs, the authors hope the demonstrated transparency benefit will push users to demand logprob support, especially for open-weight models.

• Defend against evasion and obstruction: Open questions remain about providers identifying monitoring queries, caching logprobs, or refusing short-output requests; the authors argue these tactics risk creating detectable inconsistencies elsewhere.

• Handle changes invisible in the first token: Modifications such as adjusting the bias toward the end-of-sequence token to reduce verbosity could leave initial tokens unchanged, though the authors argue the vast majority of realistic changes should appear in the first output token.

Target Audience

This paper is most useful to ML systems engineers and platform teams responsible for LLM API reliability, to researchers who depend on model-version stability for reproducible experiments, to auditors and regulators performing compliance assessments of deployed models, and to LLM API providers interested in monitoring their own inference stacks. It is also relevant to readers studying non-determinism in LLM inference, since the method builds directly on the distinction between intentional temperature-based randomness and unintentional batch- and hardware-driven randomness.

Authors’ abstract

When using an LLM through an API provider, users expect the served model to remain consistent over time, a property crucial for the reliability of downstream applications and the reproducibility of research. Existing audit methods are too costly to apply at regular time intervals to the wide range of available LLM APIs. This means that model updates are left largely unmonitored in practice. In this work, we show that while LLM log probabilities (logprobs) are usually non-deterministic, they can still be used as the basis for cost-effective continuous monitoring of LLM APIs. We apply a simple statistical test based on the average value of each token logprob, requesting only a single token of output. This is enough to detect changes as small as one step of fine-tuning, making this approach more sensitive than existing methods while being 1,000x cheaper. We introduce the TinyChange benchmark as a way to measure the sensitivity of audit methods in the context of small, realistic model changes.