Large Language Model Reasoning Failures

AI summary

Overview

Research area: Artificial intelligence — specifically the reasoning capabilities and systematic weaknesses of large language models (LLMs).

Technical level: Intermediate. The paper is a survey, so it is accessible to readers with a general grounding in machine learning, but it references a very large body of specialized literature on cognition, logic, and embodied AI.

Scope: The paper unifies scattered research on why LLMs fail at reasoning into a single taxonomy spanning reasoning types (embodied vs. non-embodied; informal vs. formal) crossed with failure types (fundamental, application-specific, robustness).

What This Paper Is About

LLMs achieve impressive results on reasoning benchmarks, yet they still fail in ways that seem trivial to humans — reversing facts, counting, tracking beliefs, or solving mildly reworded problems. Prior work documented these failures case by case, but no single framework connected them or explained their shared roots. This paper is the first comprehensive survey dedicated to cataloging, classifying, and explaining LLM reasoning failures, and it organizes the field around a two-axis taxonomy that links each failure to its causes and possible mitigations.

Key Contributions

1. A two-axis taxonomy for reasoning failures. The authors classify reasoning by type (embodied vs. non-embodied, with the latter split into informal/intuitive and formal/logical reasoning) and classify failures along a separate axis (fundamental architectural failures, application-specific limitations, and robustness issues). Each failure is positioned at the intersection of the two axes.

2. A systematic review across every reasoning domain. The survey covers individual cognition (working memory, inhibitory control, cognitive flexibility), social reasoning (Theory of Mind, moral judgment, multi-agent coordination), formal logic (reversal curse, compositional reasoning, syllogisms), mathematics (counting, arithmetic, word problems), and embodied reasoning (1D text, 2D perception, 3D physical settings).

3. Root-cause analysis and mitigation mapping. For each failure category, the paper traces underlying causes — tokenization, positional encoding, causal masking, the next-token-prediction objective, RLHF, lack of embodiment — and surveys mitigation strategies organized as data-centric, in-processing, and post-processing approaches.

4. A public research repository. The authors release a continuously updated GitHub collection of works on LLM reasoning failures to serve as an entry point for the field.

Main Findings

• Failures cluster into three categories. Fundamental failures are intrinsic to LLM architectures and affect almost all downstream tasks; application-specific limitations appear in particular domains despite expected competence; robustness issues show up as inconsistent performance under logically irrelevant variations.

• Cognitive biases are inherited, not just learned. LLMs exhibit order bias, anchoring, framing effects, confirmation bias, and the "content effect." The paper attributes these to three sources: biased patterns in pretraining text, architectural asymmetries such as causal masking, and RLHF aligning models with biased human raters.

• Theory of Mind remains brittle. Early models failed basic false-belief tasks; newer models solve many standard tests but collapse under minor rephrasing. Higher-order tasks — predicting behavior, moral judgment, translating understanding into action — remain unsolved.

• The reversal curse reveals a structural asymmetry. Models trained on "A is B" frequently cannot answer "B is A." The paper links this to unidirectional training objectives that induce weight asymmetry, and notes that scaling alone will not fix it due to Zipf's law. BERT-style bidirectional models are largely unaffected.

• Compositional reasoning fails even when components succeed. LLMs can solve individual facts or problems but fail when they must be combined, with performance degrading as compositional depth increases or distractors are added.

• Counting and arithmetic fail for architectural reasons. Tokenization, positional encoding, and training data composition — not prompting — are the primary culprits. Models often fail easier tasks (last-digit arithmetic) while succeeding at harder ones (first-digit identification), indicating heuristic pattern-matching rather than genuine computation.

• Embodied reasoning is deeply deficient. LLMs lack physical grounding, leading to errors in object attributes, spatial relations, physical laws, and affordances. These failures persist across text-based, vision-based, and real-world physical tasks.

• Multi-agent systems compound individual failures. Long-horizon planning collapses, communication breaks down due to weak Theory of Mind, and errors cascade when systems lack verification and termination checks.

• Mitigations rarely generalize. Prompt engineering, fine-tuning, activation steering, and adversarial training produce improvements in narrow contexts, but biases and reasoning failures often re-emerge when contexts shift.

Methodology in Plain English

This is a survey, not an experimental study, so the "method" is organizational and analytical rather than empirical. The authors first construct a taxonomy of reasoning by drawing on cognitive science and philosophy — separating embodied from non-embodied reasoning, and splitting non-embodied reasoning into intuitive (informal) and rule-based (formal) types. They then construct a parallel taxonomy of failure types. Using both axes, they systematically review existing literature, assigning each documented failure to a cell in the framework. For every failure, they describe what it looks like, cite the studies documenting it, trace its likely root causes, and outline the mitigation strategies that have been tried. Representative failure examples are collected in an appendix, and the full bibliography is published as a living GitHub repository.

Why This Matters

Impact on research: The paper reframes reasoning failures from isolated curiosities into a coherent research area with shared root causes. By showing that many failures trace back to the same architectural and training limitations, it gives researchers a map for where interventions are most likely to generalize rather than patch a single symptom.

Real-world applications:

• Scientific and medical decision support — reasoning failures in arithmetic, causal inference, and compositional reasoning directly undermine reliability in high-stakes analytic tasks.
• Multi-agent automation — planning systems that coordinate multiple LLM agents need the long-horizon planning and communication robustness this paper shows are currently missing.
• Customer-facing assistants — cognitive biases like framing effects and order bias cause inconsistent answers to equivalent questions, eroding user trust.
• Safety-critical and ethical deployment — inconsistent moral reasoning and susceptibility to jailbreaks limit use in domains governed by norms and regulations.

Industry relevance: Any organization deploying LLMs for reasoning-heavy workloads — code generation, financial analysis, legal reasoning, robotics — should treat the failure modes cataloged here as known limitations rather than surprises. The robustness findings in particular argue for dynamic evaluation rather than one-time benchmark scores, since small adversarial variations can produce large accuracy drops.

Future Directions

• Move from prompting to architectural fixes. The paper repeatedly notes that prompt-level interventions produce surface improvements. Open question: which failures require changes to attention mechanisms, positional encoding, or training objectives rather than better prompts?

• Build grounded reasoning systems. Since many failures trace to the absence of embodiment and experiential feedback, a key question is whether multimodal grounding, simulation, or neuro-symbolic methods can supply the missing physical and causal understanding.

• Develop dynamic, logic-preserving benchmarks. Static benchmarks overstate capability. The survey points toward benchmarks that systematically vary surface features while preserving task logic, and that evaluate reasoning steps rather than only final answers.

• Establish generalizable mitigation frameworks. Current mitigations are domain-specific and fragile. A central open problem is whether there is a unified approach to reduce biases, improve Theory of Mind, and stabilize multi-agent coordination without task-by-task engineering.

Target Audience

This paper is most valuable for AI researchers studying reasoning, LLM evaluation, or interpretability; for engineers who need a reference catalog of known failure modes before deploying reasoning systems; and for cognitive scientists and psychologists interested in the parallels between LLM failures and human cognitive biases. Graduate students entering the field will find the taxonomy and GitHub repository especially useful as an orientation to the literature.

Authors’ abstract

Large Language Models (LLMs) have exhibited remarkable reasoning capabilities, achieving impressive results across a wide range of tasks. Despite these advances, significant reasoning failures persist, occurring even in seemingly simple scenarios. To systematically understand and address these shortcomings, we present the first comprehensive survey dedicated to reasoning failures in LLMs. We introduce a novel categorization framework that distinguishes reasoning into embodied and non-embodied types, with the latter further subdivided into informal (intuitive) and formal (logical) reasoning. In parallel, we classify reasoning failures along a complementary axis into three types: fundamental failures intrinsic to LLM architectures that broadly affect downstream tasks; application-specific limitations that manifest in particular domains; and robustness issues characterized by inconsistent performance across minor variations. For each reasoning failure, we provide a clear definition, analyze existing studies, explore root causes, and present mitigation strategies. By unifying fragmented research efforts, our survey provides a structured perspective on systemic weaknesses in LLM reasoning, offering valuable insights and guiding future research towards building stronger, more reliable, and robust reasoning capabilities. We additionally release a comprehensive collection of research works on LLM reasoning failures, as a GitHub repository at https://github.com/Peiyang-Song/Awesome-LLM-Reasoning-Failures, to provide an easy entry point to this area.