The Pulse
UCR Researchers Separate LLM Confidence From Correctness
UC Riverside researchers identified distinct internal features associated with language models’ confidence and correctness, and tested how modifying those features affected answers.
AI.info Team ·
“A model can answer with certainty, but also be wrong, or it can answer while being less confident and can be correct.”
Het Patel, UCR computer science doctoral student and lead author, quoted by UC Riverside News
Confidence does not certify a correct answer
UC Riverside publicized the research on September 23, describing a study that finds large language models’ confidence and correctness can be tied to distinct internal features. The researchers examined Meta’s Llama-3.1-8B and Google’s Gemma-2-9B, using sparse autoencoders to identify patterns in each model’s internal activity.
The paper, submitted to arXiv on April 21, divides model responses into four groups: confident and correct, confident and incorrect, uncertain and correct, and uncertain and incorrect. That framework lets the team examine signals associated with confidence separately from those associated with answer correctness, rather than treating a model’s expressed uncertainty as a reliable proxy for whether it knows the answer.
Three kinds of features behaved differently
The analysis identified features associated mainly with uncertainty, features associated mainly with incorrect answers, and a third group linked to both. The researchers then suppressed selected features while the already-trained models answered questions. Turning down uncertainty-only features sharply hurt accuracy, while suppressing most incorrectness-only features had little effect.
Suppressing the overlapping, or “confounded,” features produced a different result: accuracy improved by as much as 1.1%, and output entropy fell by as much as 75%. The authors report that effects transferred across the ARC-Challenge and RACE question-answering benchmarks. The intervention happens during inference, as a model generates answers; it does not require retraining the model.
Three Llama features helped flag when to abstain
In a separate test, activations from three confounded features in one middle layer of Llama-3.1-8B helped predict whether an answer would be incorrect. When the model abstained on questions flagged by that signal, its accuracy on the questions it did answer rose from 62% to 81%, at 53% coverage. In the researchers’ comparison, letting the model decide for itself when to say “I don’t know” raised accuracy only to about 64%.
Those results describe a trade-off: the higher accuracy came with the model answering only about half the questions. The feature signal may therefore help identify risky answers, but it does not make the model uniformly more reliable across every prompt.
The results point to a targeted intervention, not a general fix
The findings offer a way to probe and alter specific internal patterns without changing model weights through another training run. But the experiments cover two open-weight models and question-answering benchmarks; the paper does not establish that the same features or gains will carry over to other models or real-world tasks.
The study, “Are LLM Uncertainty and Correctness Encoded by the Same Features? A Functional Dissociation via Sparse Autoencoders,” was accepted for presentation at the SIAM International Conference on Data Mining in Salt Lake City in November, according to UCR. Its central result is narrower than a general solution to confident errors: some internal signals track uncertainty, some track mistakes, and manipulating the signals linked to both changed how the tested models answered.