Beyond Hallucinations: A Composite Score for Measuring Reliability in Open-Source Large Language Models

AI summary

Overview

• Research area: Natural Language Processing / evaluation methodology for open-source Large Language Models (LLMs), specifically reliability measurement combining calibration, robustness, and uncertainty quantification.
• Technical level: Intermediate. The paper uses standard reliability metrics (ECE, Brier Score, NLL, AUROC) but explains each normalization step in plain formulas, so readers with basic ML familiarity can follow it; some background in calibration and uncertainty estimation helps.
• Scope: The paper proposes the Composite Reliability Score (CRS), a single weighted metric aggregating calibration, robustness, and uncertainty, and applies it to ten open-source LLMs across five question-answering datasets. arXiv:2512.24058v1 [cs.CL], 30 Dec 2025, licensed CC BY 4.0, with code at https://github.com/rohitsalla/CRS.git.

What This Paper Is About

Open-source LLMs such as LLaMA, Mistral, and Gemma are increasingly deployed in decision-critical domains including healthcare, law, and finance, but existing evaluations measure only isolated aspects of reliability. Accuracy, BLEU, or single reliability metrics give fragmented insight and can hide weaknesses such as overconfident errors, brittleness under input shifts, and unreliable uncertainty estimates. The goal of this paper is to build one unified, interpretable score that captures all three reliability dimensions at once and to use it to rank ten leading open-source models.

Key Contributions

1. A unified reliability metric (CRS): A single score integrating calibration, robustness, and uncertainty quantification into one interpretable number, with each component normalized to [0, 1] so higher always means better.
2. A large-scale evaluation: Ten open-source LLMs evaluated across five QA datasets under baselines, three controlled input perturbations, two uncertainty estimation methods, and two post-hoc calibration techniques.
3. Stability and sensitivity analysis: Alternative weight configurations, bootstrap confidence intervals, and leave-one-out dataset analysis used to test whether CRS rankings hold up.
4. Diagnostic insight into failure modes: Evidence that accuracy alone is not a proxy for reliability, and that composite scoring reveals weaknesses missed by single metrics.

Main Findings

• Mistral-8x22B ranks highest overall: CRS of 0.81, placing it alone in the "High" reliability tier (scores ≥ 0.8), driven by the strongest performance across all three pillars (Calibration 0.91, Robustness 0.78, Uncertainty 0.73).
• Most models land in the "Moderate" tier: Qwen3-235B (0.76), DeepSeek R1 0528 (0.75), Llama 4 Scout (0.72), MiniMax-Text-01 (0.71), Gemma 2 (0.70), Kimi K2 (0.67), and Mistral-7B (0.63) fall between 0.6 and 0.8.
• 7B models rank lowest: LLaMA-3-7B scores 0.57 and Falcon-7B scores 0.52, both in the "Low" tier (below 0.6), indicating limited reliability and unsuitability for safety-critical environments.
• Calibration is not determined by size: Mistral-8x22B achieves the best baseline calibration (ECE 0.031, Brier Score 0.128, NLL 0.332), while Falcon-7B is worst calibrated (ECE 0.062, Brier Score 0.179, NLL 0.566), reflecting strong overconfidence. The paper states model size alone does not determine calibration quality.
• Adversarial inputs hurt most: Adversarial perturbations cause the largest average performance loss at 11.2 percent. Mistral-8x22B and DeepSeek R1 show the strongest robustness with 6–7 percent degradation, while Falcon-7B and LLaMA-3-7B exceed 10 percent drops.
• Ensembles beat MC Dropout for every model: All ten models used the Ensemble estimate as their best uncertainty method. AUROC for error detection ranges from 0.882 (Mistral-8x22B) down to 0.716 (Falcon-7B); Mistral-8x22B, DeepSeek R1 0528 (0.878), and Qwen3-235B (0.872) approach 0.90, while the 7B models stay below 0.75.
• Post-hoc calibration helps all models: Isotonic regression yields the strongest gains. LLaMA-3-7B improves from ECE 0.057 to 0.046, and Mistral-8x22B improves from 0.031 to 0.025; temperature scaling helps but less than isotonic regression.
• Rankings are stable under weight changes: A calibration-focused weighting (0.5, 0.25, 0.25) and a robustness-focused weighting (0.2, 0.5, 0.3) both preserved the ordering of the top and bottom three models.
• Small CRS gaps are not meaningful: Bootstrap confidence intervals over 100 samples showed CRS variance below 0.02 for high-ranked models, so differences such as 0.75 versus 0.76 are not statistically meaningful.
• Results are not dataset-specific: Leave-one-out analysis shows an average deviation below 0.03 and no model changes reliability tier.
• High accuracy is not reliability: LLaMA-3-7B achieves strong clean accuracy yet shows poor calibration and weak uncertainty estimates, and Mistral-7B and LLaMA-3-7B have comparable robustness but differ substantially once calibration and uncertainty are included.

Methodology in Plain English

The authors define reliability as three pillars and score each one on a 0-to-1 scale where higher is better.

Calibration. They measure Expected Calibration Error (ECE), which bins predictions by confidence and compares mean confidence against actual accuracy. To turn "lower ECE is better" into "higher score is better," they divide the model's ECE by the worst ECE observed among all baseline models and subtract that ratio from 1, flooring at 0.

Robustness. They apply three perturbations to every dataset: typographical noise by swapping characters within words at a fixed rate of 5% of tokens; paraphrasing through back-translation with MarianMT (English–German–English); and adversarial rewrites generated with TextFooler, which replaces key tokens using embedding-based synonym selection. The robustness score is the fraction of clean accuracy retained, i.e., 1 minus the average accuracy drop divided by average clean accuracy.

Uncertainty. They estimate predictive uncertainty two ways. MC Dropout enables dropout with probability 0.1 at inference and runs 10 stochastic forward passes, using the variance across predicted probabilities as the uncertainty signal. Ensembles combine three models built from checkpoints trained with different random seeds from the same model family, using prediction variance as the estimate. Quality is judged by AUROC for separating correct from incorrect predictions, mapped linearly to a 0–1 score by (AUROC − 0.5) / 0.5. For each model the paper reports the better of the two methods.

Combining them. The final score is a weighted sum of the three components, with balanced weights of one third each for general evaluation. The paper also tests the two alternative weight configurations described above. Scores of 0.8 or higher mean high reliability suitable for deployment with minimal supervision; 0.6 to 0.8 means moderate reliability suitable with human oversight; below 0.6 means limited reliability and unsuitability for safety-critical use.

Models and data. The evaluated models are LLaMA-3-7B, Mistral-7B, Falcon-7B, Kimi K2 (15B), Llama 4 Scout (17B), Mistral-8x22B, Qwen3-22B, MiniMax-Text-01 (25B), Gemma 2 (27B), and DeepSeek R1 (27B). The datasets are TriviaQA, NaturalQuestions, SQuAD 2.0, MedQA, and ARC, covering general knowledge, reading comprehension, medical reasoning, and multi-step reasoning. Baselines measured on clean test sets include ECE, Brier Score, and NLL; calibration interventions tested are temperature scaling (a single scalar learned on a held-out validation set that rescales logits) and isotonic regression (a monotonic mapping between predicted confidence and accuracy). Note that the experimental setup section names one model "Qwen3-22B" while the results tables list "Qwen3-235B."

Why This Matters

• Research impact: The paper argues that calibration, robustness, and uncertainty research streams remain siloed, and that aggregated benchmarks such as GLUE and SuperGLUE measure accuracy but neglect reliability. CRS offers a shared, reproducible scoring frame (with public code) for comparing models on reliability rather than correctness alone, and its stability checks set a template for validating composite metrics.
• Real-world applications:
  • Healthcare decision support: models used for medical reasoning tasks such as those in MedQA, where the paper suggests calibration may deserve more weight.
  • Legal and financial advisory tools: decision-critical settings where overconfident errors and unreliable uncertainty estimates are costly.
  • Adversarial or noisy deployments: environments where inputs contain typos or deliberate manipulation, where the paper suggests robustness should be weighted more heavily.
  • Deployment gating: the three-tier interpretation (≥0.8 deploy with minimal supervision, 0.6–0.8 use with human oversight, <0.6 unsuitable for safety-critical work) offers a concrete way to route models to appropriate levels of human review.
• Industry relevance: The finding that ensembles outperform MC Dropout for every model gives practitioners a concrete guidance point for uncertainty estimation. The finding that smaller 7B models cluster at the bottom on calibration and uncertainty, while model size alone does not determine calibration, informs model selection and suggests post-hoc calibration (especially isotonic regression) as a cheap reliability improvement.

Future Directions

• Extend CRS to generative settings: The current evaluation focuses on extractive QA; generative tasks may require adapted definitions of calibration and robustness.
• Incorporate hallucination metrics directly: Hallucination behavior is not measured in this work, despite the paper's framing of moving "beyond hallucinations."
• Improve normalization and weighting: The authors describe their normalization and weighting as heuristic, noting that worst-case anchoring can amplify small gaps when ECE values are close, and suggest percentile, logistic, task-aware, or learned formulations as alternatives.
• Broaden the evaluation surface: Promising next steps named include multilingual and out-of-distribution robustness, plus integrating fairness and prompt-injection robustness into CRS.

Target Audience

Researchers and practitioners evaluating or deploying open-source LLMs will benefit most, particularly those working on model reliability, calibration, uncertainty quantification, and robustness. It is also relevant to engineering teams selecting among open-source models for regulated or decision-critical applications, and to evaluation-focused researchers who want a template for combining heterogeneous reliability metrics into a single validated score. Readers without prior exposure to ECE or AUROC will need to consult the cited background work, since the paper assumes familiarity with those measures.

Authors’ abstract

Large Language Models (LLMs) like LLaMA, Mistral, and Gemma are increasingly used in decision-critical domains such as healthcare, law, and finance, yet their reliability remains uncertain. They often make overconfident errors, degrade under input shifts, and lack clear uncertainty estimates. Existing evaluations are fragmented, addressing only isolated aspects. We introduce the Composite Reliability Score (CRS), a unified framework that integrates calibration, robustness, and uncertainty quantification into a single interpretable metric. Through experiments on ten leading open-source LLMs across five QA datasets, we assess performance under baselines, perturbations, and calibration methods. CRS delivers stable model rankings, uncovers hidden failure modes missed by single metrics, and highlights that the most dependable systems balance accuracy, robustness, and calibrated uncertainty.