A Unified Definition of Hallucination: It's The World Model, Stupid!

AI summary

Overview

Research area: Natural Language Processing / LLM evaluation and reliability (hallucination in language models) Technical level: Intermediate (conceptual position paper with formal definitions; assumes familiarity with LLM evaluation settings) Scope: The paper proposes a single unifying definition of hallucination as inaccurate world modeling, formalizes it with a reference world model, view function, conflict policy, and truth function, and shows how existing definitions and benchmarks are special cases.

What This Paper Is About

Despite years of mitigation efforts, LLMs still hallucinate, largely because researchers mean different things by the term depending on the task (translation, summarization, QA, RAG, agents, vision-language). The paper argues that all these definitions share a hidden structure—a reference "world" plus assumptions about what the model can see and how conflicts are resolved—and that spelling this structure out is the key to making hallucination research coherent and comparable.

Key Contributions

1. A unified definition of hallucination as inaccurate (internal) world modeling observable in the model's output, formalized via a reference world model W = (S, H, R), a view function V(W, x), a conflict policy P, and a truth function T_{W,P}(x, c) ∈ {true, false, unknown}.
2. A demonstration that prior definitions—source-faithfulness in translation/summarization, open-domain factuality, RAG, agentic action hallucinations, and VLM hallucinations—are all special cases obtained by varying W, V, and P.
3. A diagnostic checklist for benchmark design (specify W, expose V, state P, and make claim extraction C(y) inspectable), showing how existing benchmarks like FavaBench, HALoGEN, and HaluEval differ in how explicitly they ground these components.
4. A recipe for scalable, label-free hallucination benchmarks using synthetic environments with fully specified worlds, illustrated with a chess case study and connected to the companion HalluWorld benchmark (gridworlds, chess, terminal tasks).

Main Findings

• Hallucination is not one phenomenon but a family: Different tasks pick different reference worlds (source document, real-world facts, retrieved corpus, environment DOM), view functions, and conflict policies, yet the underlying detection rule—"some observable claim is false"—is the same.
• Assumptions are usually implicit: Most definitions do not state what counts as truth, what the model is supposed to observe, or how conflicts between sources resolve, which makes benchmarks hard to compare and "reduces hallucinations by 40%" claims hard to interpret.
• Hallucination is a subset of errors: The framework cleanly separates world-modeling errors (true hallucinations) from planning errors (correct beliefs, poor action) and incentive/reward errors (model knows it is uncertain but answers confidently). The authors align this with Yann LeCun's informal 5-fold error taxonomy.
• Abstention is not a fix for a wrong world model: Responding to the calibration view, the authors argue that a well-calibrated agent that abstains still fails the task if its internal model of the environment is incorrect—it just avoids verbalizing the error.
• Synthetic worlds enable automatic labeling: Because W, V, and T can be specified programmatically in games and simulators (e.g., chess board states and legal-move rules), hallucination labels can be derived by construction rather than by human annotators or another LLM.
• Benchmarks differ in grounding quality: For example, FavaBench makes V observable via retrieval logs, HALoGEN partially exposes V, and HaluEval leaves more ambiguity in claim extraction C(y), which can explain disagreements across benchmarks.

Methodology in Plain English

This is a position paper, not an empirical study. The authors first survey how the term "hallucination" has shifted across research generations: from translations ungrounded in the source, to summarization faithfulness (intrinsic vs. extrinsic), to broad factual correctness, to fine-grained span-level taxonomies, to agentic action-level failures, and finally to multimodal settings. They then extract the common structure behind these views and formalize it: a reference world model defines what is true; a view function defines what the model can see; a conflict policy defines which source wins when information disagrees; and a truth function labels each atomic claim as true, false, or unknown. A model hallucinates when at least one claim in its output is labeled false. They walk through four worked examples (summarization, open-domain QA, RAG, and agentic browser use) with all components spelled out, then illustrate how to build benchmarks in a chess environment where the board state, move history, and game rules provide an exact reference world.

Why This Matters

Impact on research: The framework gives a shared vocabulary for comparing hallucination benchmarks, clarifies what mitigation techniques actually target (improving V, improving T, or improving the model's internal approximation to W), and opens the door to large-scale benchmarks with programmatically determined labels instead of expensive human annotation.

Real-world applications:

• Healthcare and legal systems, where fluent but wrong explanations must be distinguished from planning failures or uncertainty.
• Retrieval-augmented enterprise assistants, where the framework clarifies when retrieved evidence should override parametric memory.
• Web and computer-use agents, where hallucinating a nonexistent button or page state causes real task failures and security risks.
• Vision-language systems such as medical imaging or autonomous driving, where hallucination is defined against a physical scene rather than a text source.

Industry relevance: Teams deploying LLMs can use the W/V/P/T checklist to audit their own evaluation pipelines, specify what "hallucination rate" means in their product, and decide whether to invest in better retrieval (view), better verification (truth function), or better model representations (world modeling).

Future Directions

• Instantiating the framework across many synthetic environments beyond chess—the paper points to HalluWorld covering gridworlds, chess, and terminal tasks as one concrete realization.
• Developing robust claim extraction methods C(y) that preserve epistemic status (e.g., distinguishing "X is true" from "X is likely true" or "X is 70% likely").
• Reconciling the world-model view with the calibration and retrieval-oriented views of hallucination, which the paper begins but does not fully resolve.
• Establishing whether mitigation gains in one setting (e.g., RAG) transfer to others once W, V, and P are made explicit, and identifying the conditions under which such transfer should be expected.

Target Audience

This paper is most useful for hallucination benchmark designers, LLM evaluation researchers, and applied scientists building reliability pipelines for deployed language models. It also benefits readers who need a conceptual map of the fractured hallucination literature, and policy or safety researchers who want precise language for distinguishing hallucinations from other model failures.

Authors’ abstract

Despite numerous attempts at mitigation since the inception of language models, hallucinations remain a persistent problem even in today's frontier LLMs. Why is this? We review existing definitions of hallucination and fold them into a single, unified definition wherein prior definitions are subsumed. We argue that hallucination can be unified by defining it as simply inaccurate (internal) world modeling, in a form where it is observable to the user. For example, stating a fact which contradicts a knowledge base OR producing a summary which contradicts the source. By varying the reference world model and conflict policy, our framework unifies prior definitions. We argue that this unified view is useful because it forces evaluations to clarify their assumed reference "world", distinguishes true hallucinations from planning or reward errors, and provides a common language for comparison across benchmarks and discussion of mitigation strategies. Building on this definition, we also connect our framework to HalluWorld, a complementary benchmark that instantiates fully specified reference world models for stress-testing model hallucinations.