Martingale Score: An Unsupervised Metric for Bayesian Rationality in LLM Reasoning

AI summary

Overview

• Research area: Evaluation of large language model (LLM) reasoning, specifically Bayesian rationality, confirmation bias, and process-based (rather than outcome-based) reasoning metrics.
• Technical level: Intermediate. The method is a linear regression, but it is grounded in Martingale theory from Bayesian statistics and assumes familiarity with LLM reasoning paradigms (chain of thought, debate) and probabilistic scoring (Brier Score).
• Scope: The paper proposes an unsupervised regression-based metric (the Martingale Score) for measuring "belief entrenchment" in LLM reasoning and applies it across three problem domains, six models, two reasoning techniques, and three prompt conditions.

What This Paper Is About

Iterative reasoning in LLMs may entrench a model's existing beliefs instead of improving its pursuit of truth, but in open-ended tasks it is hard to tell a justified update (one later supported by evidence) from a biased one, and prior work has relied on synthetic or domain-specific setups. The authors define belief entrenchment as a statistical violation of the Martingale property, which states that under rational belief updating the expected future belief should equal the current belief, so belief updates should not be predictable from the prior alone. They operationalize this as the Martingale Score, the estimated slope of a regression of belief updates on prior beliefs, and use it to detect entrenchment and to predict ground-truth accuracy where labels exist.

Key Contributions

1. The belief entrenchment problem and the Martingale Score. The authors define belief entrenchment as a statistical property, quantify confirmation bias in LLM reasoning, and introduce the Martingale Score (the predictability of belief updates based solely on the prior) as an unsupervised, domain-agnostic measure of it.
2. Uncovering widespread belief entrenchment. Using the Martingale Score, they evaluate mainstream LLMs and report entrenchment across different domains (Forecasting, r/ChangeMyView, and OpenReview), prompts (prior-conforming, no-prompt, critical-thinking), and model families (GPT, DeepSeek, Gemini, Llama).
3. Connecting belief entrenchment to accuracy loss. They find entrenchment consistently predicts an accuracy drop in domains with ground truth labels, suggesting the Martingale Score serves as a proxy for reasoning quality even where ground truth is unavailable or still unfolding.
4. Theoretical justification and judge validation. They prove (Proposition 1) that if the Martingale property holds, the population coefficient is 0 and the sample estimate is unbiased and consistent, and they show cross-judge and human-LLM agreement backing the belief-extraction procedure.

Main Findings

• Entrenchment is prevalent across setups: In most experiments, including almost all chain-of-thought (CoT) conditions (51 out of 54), the authors observe positive Martingale Scores, indicating consistent belief entrenchment. A positive score means an M-unit increase in the belief update per unit increase in the prior belief.
• Entrenchment is not an artifact of the prior-conforming prompt: Even with no system prompt and with the critical-thinking prompt, entrenchment still occurs, though to a lesser extent. The mean Martingale Scores with 95% CI are 0.082 ± 0.018 (prior-conforming), 0.075 ± 0.014 (no prompt), and 0.072 ± 0.018 (critical thinking).
• Value-laden domains show more severe entrenchment: Within the same model, system prompt, and setup, the Martingale Score is always higher in r/ChangeMyView than in Forecasting. The mean CoT scores are 0.103 ± 0.013 (ChangeMyView), 0.037 ± 0.011 (Forecasting), and 0.086 ± 0.012 (OpenReview), each with 95% CI.
• Prompt manipulation works in Forecasting but not elsewhere: Entrenchment is more severe under the prior-conforming prompt and less severe under the critical-thinking prompt in Forecasting, but this pattern does not hold in r/ChangeMyView or OpenReview.
• Entrenchment correlates with worse accuracy: The absolute value of the Martingale Score is positively correlated with the Brier Score (both lower is better) across all setups. In the CoT-on-Forecasting case, a Martingale Score of 0.0 corresponds to a Brier Score smaller than 0.25, slightly better than a random guess (0.250); at a Martingale Score of 0.04, forecasting performance is worse than a random guess. Each data point represents one setup on more than 100 questions in one problem domain.
• Model-level patterns: The abstract reports identifying models (e.g., GPT-4o), reasoning techniques (e.g., chain of thought), and domains (e.g., forecasting) that are more prone to belief entrenchment. In the reported table, GPT-4o with no prompt on OpenReview-debate has a score of +0.1891 (significant at p < 0.05), and Llama 4 Maverick with the critical-thinking prompt on OpenReview-debate reaches +0.2521 (significant).
• Judge reliability holds up: All judges show a large positive correlation with GPT-4o. Human Evaluator 1 leads with Pearson r = 0.8822 and Spearman ρ = 0.8770 (195 belief samples, 20 problems), followed by DeepSeek-v3 (r = 0.7774, 24,921 belief samples), GPT-4.1-mini (r = 0.7581), Gemini-2.5-pro (r = 0.7460), and Human Evaluator 2 (r = 0.7152), all with p < 0.001. As a reference, the NeurIPS 2021 review consistency experiment showed r = 0.58 between accept decisions made by two independent committees.
• One domain does not show the accuracy link: The authors state that in OpenReview they did not demonstrate a correlation between the Martingale Score and the Brier Score, which they attribute to community-voted acceptance decisions being noisy.

Methodology in Plain English

The authors treat the start of a model's output as its "prior belief" and the end of its output, after extended reasoning or engagement with evidence, as its "posterior belief." Because LLM confidence scores (token probabilities or self-reports) are poorly calibrated, they use a separate judge model (GPT-4o in the main experiments) to read each reasoning step and assign an expressed belief score b between 0 and 1.

The Martingale Score is then the ordinary least squares slope from a linear regression of the belief update (posterior minus prior) on the prior belief. If updates were rational and Martingale-consistent, the expected update would be 0 for every prior value, the true slope would be 0, and the prior would be statistically exogenous to the update. A positive slope means updates are systematically predictable from the prior. The authors chose the slope over alternatives such as the R² of a logistic regression (which they say introduces confounders like the intrinsic variance of belief update) or logistic regression on the binary direction of update (which ignores magnitude and, they report, produced random-seeming results). Ordinary least squares also allows significance testing (t-test, p < 0.05).

Domains were selected against three criteria: questions not solvable by memorization, the presence of new evidence that could shift beliefs, and ground truth becoming available after the models' knowledge cutoff. The chosen domains are event forecasting questions from Metaculus and Polymarket; value-laden questions from the r/ChangeMyView subreddit; and ICLR submission data from OpenReview, where the model acts as an area chair deciding acceptance from the abstract, reviews, and rebuttals. Two reasoning techniques are compared: chain of thought and debate, where one model argues with its own clone. Models evaluated are GPT-4o, DeepSeek R1, DeepSeek V3, Gemini 2.0 Flash, LLaMA 4 Scout, and LLaMA 4 Maverick. System-prompt conditions are no prompt, a prior-conforming prompt that tells the model to fixate on its prior belief, and a critical-thinking prompt that encourages challenging that prior belief.

Why This Matters

Research impact. The paper shifts attention from outcome-based evaluation (which shows that reasoning helps but not how, and therefore says little about out-of-distribution generalization) to a process-based metric that can be computed without ground truth. By grounding belief entrenchment in the Martingale property of Bayesian statistics rather than in task-specific psychological measures, it gives the field a statistical definition of confirmation bias in LLM reasoning and a result showing that the metric tracks accuracy where accuracy is measurable.

Real-world applications.

• Event forecasting and prediction markets: the Martingale Score can flag models whose probabilistic predictions are anchored to their priors rather than to incoming evidence, which matters because the paper reports forecasting performance worse than a random guess at moderate entrenchment levels.
• AI-assisted scientific peer review: models acting as reviewers or area chairs may lock onto an initial impression from an abstract despite later critiques and rebuttals; the authors note entrenchment is more severe in OpenReview and r/ChangeMyView, where human judgment is required.
• Value-laden or controversial advice: in domains like r/ChangeMyView, where the ground truth is ambiguous, the metric still applies, giving a way to audit whether a model genuinely engages with counterarguments rather than restating its stance.
• Human-AI feedback loops: entrenchment in models that users consult could reinforce users' own biases, and the paper links such loops to polarization, misinformation, and collective lock-in of false beliefs.

Industry relevance. Entrenchment signals an unjustified sense of confidence that can mislead users, and the authors frame the Martingale Score as a possible training objective and as a candidate evaluation metric for reasoning pipelines, including agentic systems where deciding what information to seek depends on how the model interprets its current situation.

Future Directions

• Reinforced reasoning remains untested: The authors state that, due to resource constraints, they did not systematically study belief entrenchment in reinforced reasoning, which they call a recent and popular approach to LLM reasoning.
• External evidence seeking: The paper covers irrational engagement with the internal reasoning process only. A future component could evaluate whether models update beliefs in response to new external evidence, with the difficulty that evidence searching must not give the model direct access to ground truth.
• Validity in open-ended domains: Where ground truth is available (Forecasting, but not OpenReview), causality between entrenchment and accuracy was demonstrated; validating the Martingale Score in open-ended domains would require showing other downstream consequences of entrenchment.
• Broadening and training use: The pipeline could be extended to sycophancy, group conformity, and inverse scaling, and the metric could be converted into a training objective. If Martingale training reduces both the Martingale Score and the Brier Score in a domain, that would be new evidence that the score evaluates reasoning and generalizes beyond forecasting. Applications to measuring and mitigating entrenchment or polarization in human-AI systems such as recommender systems are another direction.

Target Audience

Researchers and practitioners working on LLM reasoning evaluation, calibration, and truthfulness; AI safety and alignment researchers interested in measuring confirmation bias and sycophancy; evaluation engineers who need process-based metrics for domains without ground truth; and social-computing or human-AI interaction researchers concerned with belief entrenchment, polarization, and misinformation in human-AI feedback loops. Readers without a statistics background will need to engage with the regression formalization and the Martingale property, but the experimental design and the accuracy-correlation results are accessible without it.

Authors’ abstract

Recent advances in reasoning techniques have substantially improved the performance of large language models (LLMs), raising expectations for their ability to provide accurate, truthful, and reliable information. However, emerging evidence suggests that iterative reasoning may foster belief entrenchment and confirmation bias, rather than enhancing truth-seeking behavior. In this study, we propose a systematic evaluation framework for belief entrenchment in LLM reasoning by leveraging the Martingale property from Bayesian statistics. This property implies that, under rational belief updating, the expected value of future beliefs should remain equal to the current belief, i.e., belief updates are unpredictable from the current belief. We propose the unsupervised, regression-based Martingale Score to measure violations of this property, which signal deviation from the Bayesian ability of updating on new evidence. In open-ended problem domains including event forecasting, value-laden questions, and academic paper review, we find such violations to be widespread across models and setups, where the current belief positively predicts future belief updates, a phenomenon which we term belief entrenchment. We identify the models, reasoning techniques, and domains more prone to belief entrenchment. Finally, we validate the Martingale Score by showing that it predicts ground-truth accuracy on problem domains where ground truth labels are available. This indicates that, while designed as an unsupervised metric that operates even in domains without access to ground truth, the Martingale Score is a useful proxy of the truth-seeking ability of a reasoning process.