Causal Strengths and Leaky Beliefs: Interpreting LLM Reasoning via Noisy-OR Causal Bayes Nets

AI summary

Overview

• Research area: Causal reasoning in large language models and humans, evaluated with computational cognitive models (causal Bayes nets), positioned at the intersection of cognitive science and LLM interpretability. Presented at the WiML Workshop at NeurIPS 2025 (arXiv:2512.11909v1 [cs.AI], 10 Dec 2025) by Hanna Dettki (New York University).
• Technical level: Intermediate. Readers need some familiarity with causal graphs (colliders), noisy-OR parameterizations, explaining away, and model-fit metrics such as LOOCV-R² and AIC; the paper's own framing is diagnostic rather than architectural.
• Scope: One sentence — the paper fits a leaky noisy-OR causal Bayes net to 20+ LLMs' and humans' probability judgments on eleven semantically meaningful collider-graph tasks, comparing Direct versus Chain-of-Thought prompting.

What This Paper Is About

Causal reasoning is often treated as a pivotal aspect of intelligence, so comparing how LLMs and humans behave on the same causal tasks should reveal each one's strengths and weaknesses. This paper asks whether LLMs are aligned with humans on identical tasks (Q1), whether each reasoner is internally consistent at the task level (Q2), and whether LLMs and humans have distinct, identifiable reasoning "signatures" (Q3). To answer these, it models probability judgments with a leaky noisy-OR causal Bayes net over a collider graph (C1 → E ← C2), separating the strength of the two causes, the prior on the causes, and a "leak" term that captures belief in unmentioned causes.

Key Contributions

1. A leaky noisy-OR CBN lens on LLM judgments. The paper models agents' judgments with parameters θ = (b, m1, m2, p(C)) ∈ [0,1], including a shared prior p(C), and selects the winning model via AIC between a 3-parameter symmetric variant (m1 = m2) and a 4-parameter asymmetric variant (m1 ≠ m2).
2. A much larger evaluation set. Relative to Dettki et al. (2025), the work replaces the logistic link with leaky noisy-OR, expands the number of evaluated LLMs by roughly 5× (20+ LLMs), and enables clearer evaluations of explaining away (EA) and Markov-violation (MV) diagnostics.
3. Three linked metrics across Direct and Chain-of-Thought prompting. Human-LLM alignment measured by Spearman correlation ρ (Q1), task-level LOOCV-R² from CBN fits (Q2), and parameter-signature profiling of (b, m1, m2, p(C)) (Q3).
4. Explicit behavioral diagnostics for EA and MV. EA is defined on the collider as Pr(C1=1 | E=1, C2=1) < Pr(C1=1 | E=1, C2=0) — a positive slope in Fig. 1(b) — and MV is defined as one cause's presence affecting belief in the other, operationalized as |IV − V| > 0 + ε with ε = 0.05.

Main Findings

• State-of-the-art models set an alignment ceiling under Direct prompting. Recent top-performing models such as gemini-2.5-pro already show strong human alignment under Direct prompting (ρ ≈ 0.85), with little to no improvement from Chain of Thought.
• CoT rescues lighter and older models. For example, gemini-2.5-flash-lite improves by +0.503 under CoT, reaching ρ = 0.845, converging toward the same ceiling.
• Humans are highly consistent reasoners; CoT improves consistency most where it is weakest. CoT raises the median LOOCV R² from 0.933 to 0.941 (+0.008, +0.91%). The lower tail rises sharply: the minimum R² goes from 0.277 (gemini-2.5-flash-lite; numeric) to 0.692 (claude-3-haiku-20240307; CoT), and the spread tightens from an IQR of 0.116 to 0.060.
• Humans sit close to the best models on consistency. Humans reach LOOCV R² = 0.937, only a narrow gap from SOTA models achieving LOOCV R² of up to 0.99 (gemini-2.5-pro).
• Explaining away is common in LLMs and often exceeds human levels. 27/30 agents show EA > 0, and 24/30 exceed the human EA level of EA_human = 0.09.
• CoT effects on EA are mixed. CoT helps agents that lack EA (for example claude-3-haiku and gemini-2.5-flash-lite) but can slightly reduce EA in strong agents (for example gpt-4o and o3-mini).
• Markov violations follow a similar mixed pattern. Eight agents show MV under Direct prompting; CoT improves most but can worsen others (for example claude-3.5-haiku).
• Parameter signatures separate clean from leaky reasoners. High-EA–no-MV agents have low leakage b (0–0.1), strong causal strength m1, m2 (0.75–0.99), and midrange priors, whereas agents with MV or weak EA show higher b (0.15–0.62) and weaker mi (0.25–0.82).
• Individual model examples are stark. Fig. 1(a) shows Markov violations for humans and gemini-2.5-flash-lite, while o3 shows no Markov violations and perfect independence of causes; Fig. 1(b) shows o3 with perfect EA, gemini-2.5-flash-lite with no EA, and humans with weak EA.

Methodology in Plain English

The researchers gave agents eleven causal inference tasks (labeled I–XI) built on a single collider structure, C1 → E ← C2, using semantically meaningful (RW17) content and numeric prompts. Agents judged the likelihood that a queried node has value 1, expressed in {0,100}, once under Direct prompting (a one-shot number) and once under Chain of Thought (think first, then answer). The same tasks were given to humans and to 20+ LLMs (GPT-5 family naming convention: gpt-5<suffix>_v<verbosity-level>_r<reasoning-effort>). Each agent's judgments were then fitted with a leaky noisy-OR causal Bayes net, whose parameters are a leak b, two causal strengths m1 and m2, and a shared prior p(C). An AIC comparison decided whether the symmetric three-parameter model (m1 = m2) or the asymmetric four-parameter model (m1 ≠ m2) won for a given agent. From these fits the authors derived three views: how well each agent correlates with humans (Spearman ρ), how well the CBN predicts held-out task judgments (LOOCV-R²), and each agent's parameter signature, which they used to diagnose explaining away and Markov violations.

Why This Matters

• Impact on research: The work shows that behavioral diagnostics (EA, MV) and generative-model parameters (leak, causal strength, prior) can be read off LLM judgments directly, offering a shared measurement scale for comparing humans and machines and an alternative to treating benchmark accuracy as the only signal. It also reframes "leakage" as a setting-dependent property rather than a pure defect.
• Real-world applications (implied by the paper's framing):
  • Auditing LLM probability estimates in decision-support tools where unexplained causes or evidence should or should not shift a belief.
  • Designing prompting strategies (Direct versus Chain of Thought) based on which models are weak, since CoT disproportionately helps the least consistent agents.
  • Model selection and monitoring across model generations, using parameter signatures rather than aggregate accuracy.
  • Comparing human and machine judgment in studies of causal cognition where the same task is administered to both.
• Industry relevance: The finding that lighter or older models converge toward a SOTA alignment ceiling under CoT is directly useful for deployment trade-offs between model cost and reasoning quality, and the leakage parameter gives teams a measurable handle on how much a model attributes outcomes to unmentioned causes — relevant for safety and calibration review.

Future Directions

1. Extend the framework to semantically meaningless tasks to test whether the observed signatures depend on content rather than structure.
2. Apply it to causal structures beyond colliders, probing reasoning robustness across graph topologies.
3. Re-examine what counts as "normative": the authors note that low-leak, strong-cause regimes are not universally optimal and depend on the user setting, since tasks that legitimately require uncertainty about unobserved causes may warrant nonzero leak. Their prompts neither instruct models to ignore nor to include unmentioned causes, leaving this dimension uncontrolled.
4. Perform a targeted analysis of the explanations received through CoT, to see whether and how LLMs represent and regulate unmentioned causes.

Target Audience

Researchers in LLM evaluation and interpretability, cognitive scientists studying causal reasoning, and practitioners who need to choose between prompting strategies or model tiers. It is most useful to readers comfortable with causal graphical models and model-comparison statistics, and less suited to those seeking an architectural or training-side explanation of the observed behavior — human–LLM alignment beyond Spearman ρ, and the exact composition of the 30 evaluated agents, are only partially reported in the available content.

Authors’ abstract

The nature of intelligence in both humans and machines is a longstanding question. While there is no universally accepted definition, the ability to reason causally is often regarded as a pivotal aspect of intelligence (Lake et al., 2017). Evaluating causal reasoning in LLMs and humans on the same tasks provides hence a more comprehensive understanding of their respective strengths and weaknesses. Our study asks: (Q1) Are LLMs aligned with humans given the \emph{same} reasoning tasks? (Q2) Do LLMs and humans reason consistently at the task level? (Q3) Do they have distinct reasoning signatures? We answer these by evaluating 20+ LLMs on eleven semantically meaningful causal tasks formalized by a collider graph ($C_1\!\to\!E\!\leftarrow\!C_2$ ) under \emph{Direct} (one-shot number as response = probability judgment of query node being one and \emph{Chain of Thought} (CoT; think first, then provide answer). Judgments are modeled with a leaky noisy-OR causal Bayes net (CBN) whose parameters $θ=(b,m_1,m_2,p(C)) \in [0,1]$ include a shared prior $p(C)$; we select the winning model via AIC between a 3-parameter symmetric causal strength ($m_1{=}m_2$) and 4-parameter asymmetric ($m_1{\neq}m_2$) variant.