SIMBA UQ: Similarity-Based Aggregation for Uncertainty Quantification in Large Language Models

AI summary

Overview

Research area: Uncertainty quantification (UQ) for large language models, specifically black-box, consistency-based confidence estimation for generative tasks. Technical level: Intermediate (familiarity with calibration metrics and basic probability helps, but the framework is described largely in plain terms). Scope: The paper proposes a general similarity-based aggregation framework, two novel instantiations from it (Bayesian aggregation and aggregation-by-classification), and evaluates them against several UQ baselines across 9 datasets spanning question answering, summarization, and text-to-SQL.

What This Paper Is About

The paper asks how to estimate the probability that an LLM's generated answer is correct using only its outputs, without access to model weights or token log probabilities. The goal is to provide well-calibrated confidence scores by aggregating pairwise similarities between multiple sampled generations, rather than by clustering outputs or asking the model to verbalize its own confidence.

Key Contributions

1. A high-level similarity-based aggregation (SIMBA) framework that unifies and generalizes consistency-based UQ approaches, positioning existing methods (such as arithmetic aggregation of pairwise similarities) as special cases of representing confidence as a function of similarities with other generations.
2. A Bayesian aggregation method that treats pairwise similarities as evidence and computes the posterior probability that a generation is correct given its similarities to other samples, requiring 5 learned parameters (a prior plus 2 parameters each for 2 Beta distributions).
3. An aggregation-by-classification method that frames confidence estimation as supervised classification, using similarity features alone (clf-pairs), mean similarity plus the generative score (clf-mean+gen), or all pairwise similarities plus the generative score (clf-pairs+gen), trained with a random forest.
4. An empirical study over 9 datasets (3 each for question answering, summarization, and text-to-SQL), including ablations on similarity metric, Rouge-L correctness threshold, choice of classifier, and number of samples.

Main Findings

• Proposed classification approaches are generally strongest. Across the metrics tested in Table 1 (accuracy from top selection, or ATS; adaptive calibration error, or ACE; and AUROC), the classification methods using similarity features performed well on all three metrics on the CoQA, SamSum, and Spider datasets.
• CoQA shows the sharpest contrast with baselines. For CoQA with LLaMA 3.3 70B, clf-pairs reached ATS 0.37, ACE 0.041, and AUROC 0.91, versus ATS 0.27, ACE 0.272, and AUROC 0.72 for the avg. log prob baseline, and ATS 0.27, ACE 0.272, AUROC 0.83 for arith-agg.
• Bayesian aggregation splits results. bayes-beta did well on ATS and AUROC but poorly on ACE on several datasets — for example, on CoQA with LLaMA 3.3 70B it reached ATS 0.38 (best in that column) and AUROC 0.90 but ACE 0.093, and on SamSum with Granite 3.1 8B its ACE was 0.911. The authors attribute this to the conditional independence assumption made for tractability possibly not holding in practice.
• Arithmetic aggregation is a strong AUROC baseline. Computing the arithmetic mean of similarities (arith-agg), which is mathematically equivalent to the spectral clustering by degree approach, performed reasonably well on AUROC but less so on other metrics.
• Spectral eccentricity performs poorly. spec-ecc was weak across the three tasks; on Spider with Codellama 34B it produced ATS 0.01, ACE 0.201, and AUROC 0.37.
• Verbalized self-assessment underperforms. p(true), which asks the model whether a generation is True or False and uses the probability of that token, showed very high ACE values, such as 1.195 on CoQA with LLaMA 3.3 70B and 1.099 on SamSum with LLaMA 3.3 70B.
• Results hold on the six additional datasets for ACE. In Table 2, clf-gen reached ACE 0.001 on Natural Questions and clf-mean+gen also 0.001; clf-pairs reached 0.003 on TriviaQA, 0.038 on CNN, and 0.054 on Spider-Realistic; clf-pairs+gen reached 0.034 on XSum; clf-mean+gen reached 0.069 on BIRD.
• Similarity metric choice changes little. Ablations comparing Jaccard, Rouge-1, and Rouge-L on CoQA, SamSum, and Spider showed the same general trends, with the proposed classification approaches comparable to each other and better than arith-agg on ACE.
• Calibration is stable across correctness thresholds. Varying the Rouge-L threshold in {0.3, 0.5, 0.7} on CoQA left trends largely the same.
• Classifier choice matters little. Random forests and logistic regression produced similar results in the ablation, though random forests were observed to perform better than logistic regression during method development.
• Supervised aggregation is more robust to sample count. Tracking ACE against the number of samples showed clf-pairs more robust than the unsupervised arith-agg.
• Token similarity metrics suit structured output. On the BIRD text-to-SQL benchmark, results indicated that text/token similarity metrics like Jaccard and Rouge-L are suitable for UQ even when the output is SQL, and that classification with random forests is a high-performing aggregator regardless of the choice of text/token metric.

Methodology in Plain English

The approach rests on the consistency hypothesis: a generation that resembles other generations sampled for the same query is more likely to be correct. The pipeline has three phases. First, the LLM generates multiple samples for a query by varying temperature — here, 5 samples at each of 6 temperatures from 0.25 to 1.5 in increments of 0.25. Second, pairwise similarities between samples are computed with a metric such as the Jaccard coefficient or Rouge-1/Rouge-L, all treated as values between 0 and 1. Third, those similarities are aggregated into a confidence score for each generation.

The authors propose four ways to do the aggregation. Simple aggregation takes the arithmetic mean of pairwise similarities. Bayesian aggregation treats each similarity as evidence and computes the posterior probability that a generation is correct, assuming similarities depend only on correctness and are conditionally independent, and modeling conditional similarity distributions with Beta distributions. Aggregation by classification trains a random forest to predict whether a generation is correct, using either all pairwise similarities, the mean similarity, or those plus the generative score (the exponentiated average logit over generated tokens). This last method is the one that goes beyond a purely black-box setting when the generative score is included.

Correctness is defined by a reward: for text-to-SQL, a generated query is correct if it returns the same result as the ground truth query when executed on the database; for QA and summarization, Rouge-L similarity above a threshold (0.5 and 0.3 respectively) counts as correct. Experiments use LLaMA 3.3 70B instruct and Granite 3.1 8B instruct for QA and summarization, and few-shot prompting with Codellama 34B instruct and Granite 34B Code instruct for text-to-SQL. Generation used a maximum of 200 new tokens, top-k = 20, top-p = 0.7, and inputs truncated to 700 tokens. Confidence is scored only on the lower 3 temperatures, since higher temperatures produce lower-quality generations. Data were split randomly into 50% train and 50% test, and each experiment was repeated 5 times, with error bars from maximum and minimum values. Calibration is measured with ACE using 5 adaptive bins.

Why This Matters

UQ methods that need only model outputs are attractive because they are robust to the constant churn of LLM releases, cheap to run, and quick to deploy at inference time. Framing confidence as a learned function of pairwise similarities also means the same machinery can be applied to short answers, long summaries, and structured outputs like SQL, which is a broader scope than methods tied to semantic similarity or natural language inference.

Real-world applications:

• Question answering systems that must decide whether to answer or escalate a query to a human.
• Summarization pipelines that flag low-confidence summaries for review before they reach a user.
• Text-to-SQL and other code generation tools where an incorrect query could execute against a production database.
• Selection among multiple candidate generations, using ATS-style confidence scores to pick one response per query.

Industry relevance: the supervised variants require only a small labeled training set and are described as lightweight and easily deployable, which matters for systems that combine several models or swap models over time. The entirely black-box variants are relevant to settings where model internals are unavailable, such as third-party or hosted APIs.

Future Directions

• Relaxing the conditional independence assumption in Bayesian aggregation, which the authors link to its weak calibration performance despite strong ATS and AUROC.
• Exploring learnable similarity functions over higher-level concepts, such as graph neural networks for complex entity matching, instead of token- or text-level metrics.
• Extending the framework to settings where correctness is ambiguous, such as when humans disagree about labels, which the authors explicitly note the current setup cannot handle.
• Applying the aggregation methods to more tasks and model families, and continuing the ablation work on similarity metrics and aggregation functions beyond the BIRD benchmark explored in the appendix.

Target Audience

Researchers and practitioners working on trustworthy AI, LLM reliability, and calibration, particularly those building black-box UQ modules or evaluating confidence estimates for generative tasks. Engineers deploying LLMs in QA, summarization, or text-to-SQL systems will find the framework practical, while readers wanting the statistical details of Bayesian aggregation or the full ablation tables will need to consult the appendix, which the truncated content does not include.

Authors’ abstract

When does a large language model (LLM) know what it does not know? Uncertainty quantification (UQ) provides measures of uncertainty, such as an estimate of the confidence in an LLM's generated output, and is therefore increasingly recognized as a crucial component of trusted AI systems. Black-box UQ methods do not require access to internal model information from the generating LLM and therefore have numerous real-world advantages, such as robustness to system changes, adaptability to choice of LLM, reduced costs, and computational tractability. In this paper, we investigate the effectiveness of UQ techniques that are primarily but not necessarily entirely black-box, where the consistency between a generated output and other sampled generations is used as a proxy for confidence in its correctness. We propose a high-level non-verbalized similarity-based aggregation framework that subsumes a broad swath of UQ approaches suitable for complex generative tasks, as well as introduce specific novel techniques from the framework that train confidence estimation models using small training sets. Through an empirical study with datasets spanning the diverse tasks of question answering, summarization, and text-to-SQL, we demonstrate that our proposed similarity-based methods can yield better calibrated confidences than baselines.