Evaluation Pitfalls and Sparsity Limitations in LLM-based Confidence Estimates for Classification

AI summary

Overview

Research area: Natural language processing — confidence estimation and selective prediction with large language models used as classifiers.

Technical level: Intermediate. The paper assumes familiarity with classification metrics such as AUROC and calibration error, but its central argument (sparse confidence scores break threshold-based metrics) is explained with clear examples.

Scope: A study showing that LLM verbalized confidence scores are extremely sparse, that this sparsity makes the choice of interpolation method in AUARC decisive for system rankings, and that a simple method called verbalization logprobs fixes the sparsity at no extra inference cost.

What This Paper Is About

When an LLM is asked to classify text and also report how confident it is, it tends to answer with only a handful of distinct numbers — for example, it says "95%" over and over. The paper shows this sparsity is a problem in practice (there are too few thresholds to choose from when deciding which predictions to reject) and a problem for evaluation (the way the accuracy-rejection curve is interpolated can flip the ranking of methods). The authors propose a fix, verbalization logprobs, which uses the model's token probabilities to turn those few discrete confidence values into a continuous, fine-grained estimate.

Key Contributions

1. Documenting the sparsity of verbalized confidence. The authors show across four datasets and two LLMs (Qwen3-32B and Claude 3.7 Sonnet) that verbalization-based methods concentrate 45–93% of their confidence values in just five distinct numbers, with Qwen3-32B producing only eight unique values on SST-2.

2. Demonstrating that interpolation choice dominates AUARC evaluation. Switching from linear (trapezoidal) to stepwise interpolation drops verbalization and sampling scores by 0.7–4.5 AUARC points and reverses rankings — most dramatically, consistency sampling moves from first to last place on Qwen3-32B, a 12.5 point absolute decrease. The authors advocate standardizing stepwise interpolation.

3. Proposing verbalization logprobs. This method computes the expected value over possible digit tokens at each position of the verbalized number, using the model's token probabilities. It reduces top-5 concentration from 92 to 1 on Qwen3-32B and achieves the best AUARC (+2.3 points over vanilla verbalization, +0.9 AUROC) without additional inference cost.

4. Establishing that the improvement is meaningful, not redundant. The correlation between verbalization logprobs and vanilla verbalization averages Spearman's ρ of 0.89 — high but not degenerate — while the number of unique confidence values increases from as few as 8 to as many as 2,227 (an 80× to 200× increase in resolution).

Main Findings

• Extreme sparsity in verbalized confidence: Qwen3-32B with vanilla verbalization produces only eight unique confidence values on SST-2 (0.6, 0.65, 0.7, 0.75, 0.85, 0.9, 0.95, 0.98), with over half of all predictions exactly 95%.

• Sparsity limits practical thresholding: In Figure 1, rejecting 205 samples yields 97.9% accuracy while rejecting 345 samples yields 98.7%. To reach at least 98% accuracy, one must reject 39.7% of predictions, versus 23.6% for 97.9% accuracy — a 16.1 percentage point jump caused by the absence of any intermediate threshold.

• Interpolation flips rankings: On Qwen3-32B, consistency sampling scores 0.808 AUARC under linear interpolation (rank 1) and 0.683 under stepwise (rank 5). Vanilla verbalization goes 0.758 to 0.713 (rank 2 to 4), Top-2 verbalization 0.757 to 0.731 (rank 3 to 2), verbalization sampling 0.741 to 0.734 (rank 4 to 1). Token logprobs is unchanged at 0.723, because its concentration in the top-5 values is 0.

• The same pattern holds for Claude 3.7 Sonnet: consistency sampling drops from 0.814 to 0.703 (rank 1 to 4), vanilla verbalization from 0.801 to 0.771 (rank 2 to 3), verbalization sampling from 0.795 to 0.790 (rank 3 to 1), and Top-2 verbalization from 0.795 to 0.775 (rank 4 to 2). Token logprobs are unavailable through the Bedrock API.

• Verbalization logprobs wins under fair evaluation: On Qwen3-32B averaged over the four datasets, it reaches 0.736 AUARC, 0.670 AUROC and 0.234 ECE with 1% top-5 concentration and a 1.7× cost, compared with vanilla verbalization at 0.713 AUARC / 0.661 AUROC / 92% top-5 concentration at the same 1.7× cost, and token logprobs at 0.723 AUARC / 0.623 AUROC at 1.0× cost.

• Comparable quality at a fraction of the cost: Verbalization logprobs (0.736 AUARC, 1.7× cost) essentially matches verbalization sampling (0.734 AUARC, 6.8× cost) because it needs a single inference call rather than four.

• Sparsity is not an artifact of the confidence range: With an alternative 0–9 confidence range, only seven unique values appear on SST-2, with most mass at 8/9 and 9/9. Qwen3-32B vanilla verbalization goes from 92% to 99% top-5 concentration (AUARC 0.758/0.713 linear/stepwise becomes 0.761/0.716), and verbalization sampling from 45% to 66% (0.741/0.734 becomes 0.754/0.728). Linear interpolation still inflates scores.

• Sparsity persists with reasoning enabled: With reasoning mode on (1024 reasoning budget tokens for Claude), top-5 concentration remains high — for Qwen3-32B, 100 for consistency sampling, 89 for vanilla verbalization, 43 for verbalization sampling, 85 for Top-2 verbalization.

• Interpolation practices are inconsistent in prior work: The authors inspected public code and found that prior work commonly uses scikit-learn's auc() function, which performs trapezoidal integration, while AUROC and AUPRC have dedicated functions (roc_auc_score() and average_precision_score()) with correct stepwise handling. There is no specialized scikit-learn function for AUARC.

Methodology in Plain English

The authors evaluate five confidence estimation methods on four classification datasets. Vanilla verbalization asks the model to state a confidence percentage from 0 to 100 alongside its label. Top-2 verbalization asks for the two best guesses with probabilities and keeps the top one. Consistency sampling runs the prompt four times and uses the frequency of the most common label as confidence. Verbalization sampling also runs four times but averages the verbalized confidence numbers. Token logprobs reads the probability the model assigned to the class-label token, without asking for a confidence statement. A random-confidence baseline establishes the lower bound.

For each method they compute AUARC — the area under the accuracy-rejection curve, where accuracy is measured only on the fraction of examples retained above a confidence threshold. The key methodological point is how the curve is drawn between observed thresholds: stepwise interpolation (a staircase) versus linear interpolation (connecting points with straight lines). With sparse confidence scores, the two produce very different areas.

Their proposed method, verbalization logprobs, keeps the vanilla verbalization prompt but replaces the sampled number with an expected value. For a two-digit percentage, it computes the confidence as the sum over digits 0–9 of 10d·P(tens digit = d) plus the sum over digits 0–9 of d·P(units digit = d), using the model's token probabilities. The authors note that strictly this expectation should be conditional on the tens digit, but standard API access only returns logprobs for the actually generated token; they treat the two positions as independent, arguing the tens digit dominates the expectation anyway.

Datasets are SST-2 (2 classes, 872 test samples), SST-5 (5 classes, 2,210 samples), Yahoo! answers topic classification (10 classes, 6,000 examples randomly sampled from 60,000 test samples, i.e. 10%), and Amazon ESCI product-query relevance (4 classes, 8,604 examples). Generations use temperature=0 unless otherwise noted, with 1000 maximum tokens; sampling-based methods use four responses at temperature 1.0. Claude runs through the Bedrock API and Qwen3-32B through vLLM. Cost is computed as the ratio of total input plus output tokens relative to a baseline that predicts only the label.

Why This Matters

Impact on research: The paper argues that a widely used metric is being computed in a way that systematically rewards sparsity-induced artifacts. Because linear interpolation assumes performance improves gradually between confidence values, and sparse methods have few values, these methods receive an artificial boost — enough to reorder leaderboards. Standardizing on stepwise interpolation makes comparisons across papers valid.

Real-world applications:

• Selective prediction and abstention: systems that route only sufficiently confident predictions to end users and send the rest to human review need finely spaced thresholds to hit a target accuracy without discarding far more data than necessary.
• Automatic evaluation and content moderation: LLM-as-judge and moderation pipelines need calibrated confidence to decide when a machine decision is trustworthy.
• Risk-coverage control in deployed classifiers: teams tuning the trade-off between coverage and error rate benefit directly from the extra resolution — the paper shows verbalization logprobs reaches ≥98% accuracy while rejecting 28.1% of examples instead of 39.7%, salvaging 11.6% of the data.
• API-constrained deployments: where vendors do not return logprobs (OpenAI, Claude, Bedrock), verbalization is often the only option, and the proposed method improves it without extra calls.

Industry relevance: The method adds no additional inference cost over vanilla verbalization (1.7× the label-only baseline versus 6.8× for verbalization sampling on Qwen3-32B) while matching sampling quality. That cost-quality trade-off matters for large-scale production classification, and the evaluation recommendation affects any team publishing or comparing confidence-estimation benchmarks.

Future Directions

• Extending the interpolation analysis to other threshold-based metrics such as AUPRC, which the authors explicitly leave for future investigation.
• Removing the digit-tokenization assumption — the method assumes each digit is a single token, which holds for the tested models but may need adaptation for other tokenizers.
• Determining the optimal number of samples for sampling-based approaches, since the paper fixes this at four based on prior evidence of early saturation, and the optimum may vary across models and tasks.
• Finding a way to reduce sparsity when token probabilities are unavailable, since verbalization logprobs requires access to logprobs and the paper offers no solution for purely black-box verbalization settings.

Target Audience

Researchers and practitioners who build or evaluate LLM-based classifiers and confidence estimators — particularly those working on selective prediction, abstention, or risk-coverage trade-offs. It is also directly relevant to benchmark maintainers and reviewers who compare confidence methods, since the central claim is that a seemingly minor implementation detail in a metric changes which method looks best. Readers need basic familiarity with classification metrics; no deep mathematical background is required to follow the main argument.

Authors’ abstract

Confidence estimation is essential when LLMs are used for classification, indicating when predictions can be trusted. However, common approaches such as verbalization produce extremely sparse outputs. For instance, Qwen3-32B verbalizes only eight unique confidence values on SST-2, with over half being exactly 95%, a pattern we observe consistently across four datasets and two LLMs. Besides limiting practical utility, we show that this sparsity critically affects evaluation: the choice of interpolation in area under the accuracy-rejection curve (AUARC) dramatically alters rankings, with consistency sampling dropping from best to worst under stepwise versus linear interpolation. We advocate for standardizing stepwise interpolation for a fairer comparison. Under such a fair evaluation, we find that weighting verbalized digits by token probabilities, a method we term verbalization logprobs, addresses sparsity and achieves the best AUARC (+2.3 points over vanilla verbalization) without incurring additional inference cost.