Grounding, Citations, and Claim-Level Support

An answer is grounded when its material claims are justified by permitted evidence under the task's source and time rules. The evidence can come from documents, databases, calculators, or other validated tools.

That definition is not this lesson's own. Rashkin and nine co-authors published it in Computational Linguistics in 2023, and they wrote it as a stipulation rather than an aspiration: “To this end, we present an evaluation framework, Attributable to Identified Sources (AIS), stipulating that NLG output pertaining to the external world is to be verified against an independent, provided source.” — Rashkin et al., “Measuring Attribution in Natural Language Generation Models”, abstract. Two words there carry the whole engineering burden. *Verified* is an act someone has to perform. *Independent* rules out the model's own account of itself.

AIS is not a slogan. It ships with a two-stage human annotation pipeline, validated on two conversational QA datasets, a summarisation dataset and a table-to-text dataset. Putting sources in the prompt does not meet that bar. The generator has to use the controlling evidence faithfully, it has to stop where the evidence stops, and something outside the generator has to check that it did.

The system should preserve where each claim came from and how it was checked. Each stage below exists so that the AIS test — verification against an independent, provided source — can actually be run afterwards, on one specific claim. The alternative is asserting it about the answer as a whole, which is not a test.

Each failure needs a different fix, and these are not categories invented for a course slide.

Three legal research products sold to professionals hallucinate between 17% and 33% of the time, despite 'hallucination-free' marketing: Lexis+ AI, Westlaw AI-Assisted Research and Ask Practical Law AI. Lexis+ AI was accurate on 65% of queries. Westlaw AI-Assisted Research was accurate 42% of the time. Ask Practical Law AI returned incomplete answers on more than 60% of queries. Magesh and five colleagues published those numbers in 2025, in the first preregistered evaluation of commercial RAG-based legal research tools.

The same paper gives the wrong-source and wrong-span failures a published name and a definition: “A response is misgrounded if key factual propositions are cited but misinterpret the source or reference an inapplicable source.” — Magesh et al., “Hallucination-Free? Assessing the Reliability of Leading AI Legal Research Tools”, §4.2 Groundedness. In their accounting, a response counts as hallucinated when it is either incorrect or misgrounded. That is the point of this section. A citation can point at a real, relevant, current document and still be wrong about what the document says.

Stale authority has been measured too, and it has a dataset. FreshQA is 600 questions deliberately sorted into never-changing, slow-changing, fast-changing and false-premise answers, scored with more than 50,000 human judgments. Vu and ten co-authors built it and published in 2024. The finding was not that some models handle time badly: “We observe that all LLMs struggle to answer questions that require fast-changing world knowledge as well as questions with false premises that need to be debunked.” — Vu et al., “FreshLLMs: Refreshing Large Language Models with Search Engine Augmentation”, abstract. Two of the four categories exist because an answer's correctness expires. Version-aware retrieval and effective dates in the citation are the fix for that column. The false-premise category is why an answer sometimes has to contradict its own question.

A response can carry several claim states at once. That is why a single verdict on a whole answer measures almost nothing.

FActScore was built on exactly that complaint. Its abstract opens on the problem this section describes: “Evaluating the factuality of long-form text generated by large language models (LMs) is non-trivial because (1) generations often contain a mixture of supported and unsupported pieces of information, making binary judgments of quality inadequate, and (2) human evaluation is time-consuming and costly.” — Min et al., “FActScore: Fine-grained Atomic Evaluation of Factual Precision in Long Form Text Generation”, abstract.

The fix is to break a generation into atomic facts and score the percentage of them supported by a reliable knowledge source. In human evaluation of people biographies generated by commercial LMs, ChatGPT achieved only 58%. The automated estimator of FActScore has an error rate of less than 2%, and it was used to evaluate 6,500 generations from a new set of 13 recent LMs — a run that would have cost $26K to evaluate by humans. The mixed states below are what a per-claim score counts. A single supported-or-not label would collapse all six into one number that names none of them.

Use representative answers and inspect both supported and unsupported material. The unit of analysis is not something this lesson is proposing. RAGTruth is a corpus of nearly 18,000 naturally generated RAG responses, annotated by hand at both the case level and the word level, with hallucination intensity, across question answering, data-to-text writing and news summarisation. Its premise is the one that makes the steps below necessary: “Despite the integration of RAG, LLMs may still present unsupported or contradictory claims to the retrieved contents.” — Niu et al., “RAGTruth: A Hallucination Corpus for Developing Trustworthy Retrieval-Augmented Language Models”, abstract.

Two things follow for your own audit. Word-level annotation over nearly 18,000 responses is what claim-level auditing looks like at scale, so the cost is known rather than speculative. And on that data a small fine-tuned model matched GPT-4 prompt-based hallucination detection, which means a labelled corpus of your own answers is worth more than a larger judge model applied to unlabelled ones.

Products that ground well tell users which claims are sourced, derived, uncertain, or unavailable. That clarity is worth more than the maximum number of links. In the audit of four generative search engines, a full-looking set of links was worth 74.5% of citations supporting their associated sentence. In tools marketed as free of the problem, retrieval still left 17% to 33% behind.

The next lesson evaluates and debugs the entire RAG pipeline. Claim support becomes one outcome among retrieval, context, latency, and abstention metrics.