E-Scores for (In)Correctness Assessment of Generative Model Outputs

AI summary

Overview

Research area: Statistical machine learning and uncertainty quantification for generative models, specifically conformal prediction and e-value methodology applied to large language model (LLM) output verification.

Technical level: Advanced. The paper builds on conformal prediction, p-values versus e-values, exchangeability, and post-hoc statistical error control. Readers need familiarity with hypothesis testing concepts to follow the theory, though the motivation and experimental section are accessible.

One-sentence scope: The paper proposes "e-scores" — e-value-based measures of incorrectness for generative model responses — that allow users to choose tolerance levels after seeing the data while still bounding a quantity called size distortion.

What This Paper Is About

Previous work uses p-value-based conformal prediction to filter LLM responses so that the probability of keeping an incorrect response is capped at a user-chosen tolerance level α. The catch is that α must be fixed before seeing the data; choosing it post-hoc invalidates the guarantee, a form of p-hacking. The authors replace p-values with e-values, defining e-scores that are low for correct responses and high for incorrect ones, and show these deliver guarantees under data-dependent ("post-hoc") tolerance levels.

Key Contributions

1. A post-hoc error framework for generative model outputs. The authors study statistical guarantees for size distortion, a post-hoc notion of error from Koning (2024), generalizing the non-post-hoc guarantees studied by Mohri and Hashimoto (2024), Cherian et al. (2024), and Rubin-Toles et al. (2025). The earlier non-post-hoc guarantee is recovered as a special case when α is a fixed pre-set value.

2. E-scores as measures of incorrectness. Built on e-values, the proposed e-scores complement responses with non-negative scores that are low for correct and high for incorrect responses. Theoretical results (Section 6.2, partially truncated in the provided text) are stated to show these achieve the size-distortion guarantee. Three variants are given based on transformations of an oracle estimator, plus a combined version obtained by averaging the underlying e-values (an admissible e-value per Vovk and Wang, 2021).

3. Broader applicability. The guarantees are shown to extend to any generative model and to a super-set of the response sets considered by the three prior works — specifically, all partial responses of all permutations of the generated response (Eq. 8), where fixing the permutation to the identity recovers the ordered-partial-response set of Eq. 1.

4. Empirical demonstration across two correctness notions — mathematical factuality and property constraints satisfaction — with comparisons against p-scores and naive transformed oracle estimators.

Main Findings

• E-scores bound size distortion; p-scores do not. The proposed e-scores reliably upper bound size distortion to 1, satisfying the desideratum in Eq. 3. p-scores are unable to achieve this; the only time they experimentally do is when all responses (correct and incorrect) are excluded, achieving 0 error by default.

• Error versus tolerance. E-scores consistently obtain a mean error lower than or approximately equal to the mean tolerance α. p-scores consistently obtain a mean error higher than the mean tolerance α.

• Precision-recall trade-off. Precision-recall curves of e-scores overlap with those of p-scores. In satisfying Eq. 3, e-scores are more conservative and prefer maintaining high precision over high recall. Restricting α to be ≤ 1 under the max-constrained adaptive strategies restricts e-score recalls relative to p-score recalls, giving partial overlap. Under fractional inclusion strategies, the e- and p-score precision-recall curves overlap completely.

• Oracle estimator choice matters. QwenPRM achieves higher precisions and recalls than MathShepherdPRM in the mathematical factuality setting (Fig. 2a), which is expected because the former is comparatively more accurate (per Zheng et al., 2025).

• Computational cost. E-scores are cheaper than p-scores in both memory and time. p-scores compute relative ranks against calibration data, requiring memory and time linear in the calibration size n for every individual test prompt-response pair. E-scores compute a sum over calibration data, requiring constant memory and time linear in n, a one-time cost amortized over all test prompt-response pairs.

• How the guarantee is achieved. Size distortion requires the expected ratio of "observing an error at α" to α to be at most 1. If this holds when α is chosen adversarially to maximize size distortion, it holds for any user choice — the worst-case strategy analyzed in Section 6.2.

Methodology in Plain English

The setup is: given a prompt, an LLM produces a response, which the authors treat as an ordered set of sub-responses. Partial responses (prefixes of that ordered set) are also valid objects to evaluate, since a response may be correct up to a point.

The guarantee machinery. Correctness is defined by an oracle that labels each response as correct (1) or incorrect (0). The authors reuse the standard guarantee — probability of any incorrect response slipping into the filtered set is at most α — but generalize it to a ratio form: the expected value of (indicator of error) divided by the (data-dependent) α is at most 1. This ratio is the "size distortion." Setting α to a fixed constant collapses this back to the original guarantee.

Two post-hoc strategies. (1) Max-constrained adaptive α: the user picks a ceiling α_max, and α becomes the maximum score in the filtered set at α_max. (2) Fractional inclusion: the user picks a fraction λ, and includes the top ⌈λ·|S|⌉ responses, with α set to the largest score among those. Setting λ = 1 gives post-hoc guarantees for the full generated response.

Constructing the scores. The e-score formula compares a test response's "value" — produced by a function f of the prompt and response — against the maximum value among incorrect calibration responses. Intuitively, f should be a proxy for the oracle: high for correct, low for incorrect. Since the true oracle is unknown, it is approximated by an estimator ô trained as a binary classifier predicting probability of correctness (trained on data independent of the test and calibration data). Three transformations of ô give three e-score variants; averaging the corresponding e-values gives a combined e-score used by default. Calibration data consists of n labeled prompts with their labeled response sets, assumed exchangeable with the test data.

Experiments. Data is split 50-50 into test and calibration sets (no training data needed, as pre-trained oracle estimators are used), and metrics are averaged over 100 random splits. An NVIDIA A100 GPU runs the pre-trained oracle estimators; remaining computations run on a CPU. The baselines are p-scores (the conformal relative-rank score), plus the raw transformed oracle estimators with no statistical guarantees.

Why This Matters

Impact on research. The work connects the rapidly growing literature on conformal prediction for LLMs to the e-value literature, whose central selling point is validity under optional stopping and post-hoc choice. It shows the standard response-filtering guarantee can be extended rather than replaced, and it expands the space of responses that can be assessed (all permutations of the generated response) beyond what prior work covers.

Real-world applications:

• Mathematical reasoning verification. Filtering chain-of-thought steps so that a user is not shown reasoning that has already gone wrong — the ProcessBench example in Fig. 1 shows e-scores jumping from 0.01 on correct steps to 4.95, 6.01, and 6.28 once the reasoning errs.
• Response quality gating in assistants. Ensuring responses meet helpfulness and instruction-following thresholds before being surfaced to a user.
• Safety and truthfulness filtering. Ensuring responses meet honesty and truthfulness criteria, relevant to deployments where factual reliability matters.
• Interactive or exploratory settings. Because α can be chosen after inspecting the scored output, an analyst can tighten the tolerance level (for example from 0.1 to 0.01) without invalidating the stated guarantees.

Industry relevance. Practitioners deploying LLMs often want to tune a confidence threshold per query rather than fixing it globally ahead of time. The amortized constant-memory e-score computation is also attractive at scale, where p-score rank computations against a calibration set become expensive per test item.

Future Directions

• Completing the practical picture. The provided text contains only Section 6.2's opening; the full worst-case analysis and theorem statements, along with the concluding section (Section 7) and appendices B and C, are not included in the truncated content. The precise conditions under which the size-distortion bound holds in full generality are therefore not reported here.
• Broader response sets. The authors note that the super-set in Eq. 8 (all partial responses of all permutations) opens "avenues for more diverse applications and use-cases," but the experiments use only the ordered prefix set of Eq. 1; evaluating the larger set empirically is a natural next step.
• Oracle estimator dependence. Results show metric sensitivity to the choice of oracle estimator (QwenPRM versus MathShepherdPRM), leaving open how best to build or select transformed oracle estimators for new domains and correctness notions.
• Beyond LLMs and beyond these two correctness notions. The paper states its guarantees hold for any generative model, but only mathematical factuality and property constraint satisfaction are tested; other modalities and other correctness definitions are untested.

Target Audience

Statisticians and machine learning researchers working on conformal prediction, e-values, and uncertainty quantification will get the most from the theoretical sections. LLM practitioners and applied scientists interested in response filtering, hallucination mitigation, or process reward models will benefit from the experimental design and the practical contrast between e-scores and p-scores. The paper assumes familiarity with conformal prediction and hypothesis testing terminology, so readers without that background should expect a steep learning curve in Sections 2, 3, and 6.

Authors’ abstract

While generative models, especially large language models (LLMs), are ubiquitous in today's world, principled mechanisms to assess their (in)correctness are limited. Using the conformal prediction framework, previous works construct sets of LLM responses where the probability of including an incorrect response, or error, is capped at a user-defined tolerance level. However, since these methods are based on p-values, they are susceptible to p-hacking, i.e., choosing the tolerance level post-hoc can invalidate the guarantees. We therefore leverage e-values to complement generative model outputs with e-scores as measures of incorrectness. In addition to achieving the guarantees as before, e-scores further provide users with the flexibility of choosing data-dependent tolerance levels while upper bounding size distortion, a post-hoc notion of error. We experimentally demonstrate their efficacy in assessing LLM outputs under different forms of correctness: mathematical factuality and property constraints satisfaction.