From Binary Groundedness to Support Relations: Towards a Reader-Centred Taxonomy for Comprehension of AI Output

AI summary

Overview

Research area: Human-Computer Interaction (HCI), specifically human-centered computing / natural language interfaces, provenance and explainability for generative AI output, with links to computational linguistics, philosophy of language, and fact-verification benchmarking. The paper carries two ACM CCS classifications: "Human-centered computing — Human computer interaction (HCI)" and "Human-centered computing — Natural language interfaces."

Technical level: Intermediate. The paper is a conceptual workshop position paper rather than a technical systems paper; it assumes familiarity with retrieval-augmented generation, groundedness/hallucination evaluation, and citation interfaces, but presents no algorithms or experimental results of its own.

Scope (one sentence): The paper proposes developing a reader-centred taxonomy of "support relations" that describes how a generated statement relates to its sources — direct quotation, paraphrase, deduction, induction, deduction contingent on assumptions — rather than only whether it is supported.

Publication venue: Workshop on Science and Technology for Augmenting Reading at CHI 2026; April 16, 2026; Barcelona, Spain. arXiv:2604.08082v1 [cs.HC], 09 Apr 2026.

Authors and affiliations: Advait Sarkar (Microsoft Research, Cambridge, UK; University of Cambridge, Cambridge, UK; University College London, London, UK), Christian Poelitz (Microsoft Research, Cambridge, UK), Viktor Kewenig (Microsoft Research, Cambridge, UK).

Keywords listed by the authors: provenance, explainability, fact verification, citation, faithfulness.

What This Paper Is About

Generative AI tools answer questions using source documents (for example through retrieval augmented generation), but current groundedness and hallucination evaluations treat the answer–source relationship as binary: the answer is either supported or unsupported. That framing hides the syntactic moves the model makes (such as direct quotation versus paraphrase) and the interpretive moves it makes (such as induction versus deduction) when it reformulates evidence into an answer. The authors argue this limits both benchmarking and the design of user-facing provenance interfaces, and they propose building an operational, reader-centred taxonomy of support relations between generated statements and source documents, along with a way to evaluate it.

Key Contributions

1. A reframing of groundedness from a binary property to a set of support relations. The authors propose determining a relation R(S, D) between a generated statement S and a source document (or set of documents) D, drawn from an explicitly defined inventory — the paper names direct quotation, paraphrase, deductive support, inductive support, and support contingent on ancillary assumptions as examples.

2. Concrete motivating examples that show why the binary framing fails. A fact-retrieval example (a fictional Acme Corporation report stating "Total revenue reached $847.2 million" in answer to a Q2 2025 revenue question) and toy sentence-level examples ("The cat sat on the mat") illustrate that equally "supported" answers differ in syntactic and epistemic status and warrant different levels of scrutiny.

3. A synthesis of theoretical starting points for the taxonomy. The paper points to Toulmin's model of argumentation (claim, grounds/data, warrant, with optional backing, qualifier, and rebuttal; Toulmin, 2003; Kneupper, 1978), work on support in argumentation systems (Cohen et al., 2014), Grice's account of conversational implicature (Grice, 1991), and scholarly-discourse representation work including the ScholOnto project (Buckingham Shum et al., 2000; Mancini and Buckingham Shum, 2006; Uren et al., 2006; Buckingham Shum et al., 2007) and the "scite" metric (Nicholson et al., 2021).

4. A proposed evaluation pathway, including a benchmark and a human annotation protocol. The authors describe an iterative design process for taxonomy construction, an annotation specification targeted at both human annotators and LLMs-as-judge (Zheng et al., 2023), a human annotation study on a stratified sample of statement–source pairs to quantify inter-annotator agreement, a "gold standard" ground truth dataset, and benchmarking of frontier models against human annotations — with existing groundedness and hallucination corpora proposed as enrichment material. The paper also states an explicit design intent, following Sarkar (2024a), that AI systems should challenge and provoke critical engagement rather than merely accelerate workflows, and that the proposed interface should not reduce to "the AI knows and tells me."

Main Findings

• The "grounded" label is too blunt even for simple fact retrieval. When the authors tested the query "What was Acme Corp's revenue in Q2 2025?" on four commercial systems, the responses were all plausibly "grounded" but differed materially in how they related to the source text: one repeated the exact figure verbatim ("$847.2 million"); another quoted the figure but added immediate surrounding context ("Total revenue reached $847.2 million"); another rounded the figure ("Total revenue was over $847 million"); and another paraphrased ("Revenue in Q2 2025 was US$ 847.2 million") in a way that fit the query more directly but quietly introduced an interpretive assumption — treating "revenue" as equivalent to "total revenue," which the paper notes may not match the user's interpretation.

• Interpretive questions make the problem unavoidable. For a question such as "Is Acme doing well?" using the same report-style excerpt, an answer cannot point to a single span: it must decide what counts as evidence (for example a revenue growth percentage, or qualitative phrases that might appear in the report such as "strong performance" and "resilience"), compress multiple sentences into a judgment, and reframe descriptive language as an evaluative conclusion.

• Sentence-level examples show distinct syntactic and epistemic statuses. Given the source "The cat sat on the mat," a "yes" answer to "Did an animal sit on the mat?" relies on the background assumption that a cat is an animal; "Is the cat able to sit?" is a deduction from the described event; and "What did the cat sit on?" could be answered with a minimal direct quote ("a mat"), a more complete quote ("The cat sat on the mat"), or a partial paraphrase ("It sat on the mat"). The paper's claim is that two answers can both be "grounded" while warranting very different levels of scrutiny — one traceable to a verbatim span, another a defensible but assumption-laden inference.

• Binary labels may interact badly with passive reliance. The authors cite prior work arguing that the greatest risk of Generative AI to knowledge work is not hallucination but the erosion of critical thinking through passive reliance (Sarkar et al., 2024), and survey evidence showing that higher confidence in AI is associated with less critical thinking and a shift from information gathering to verification (Lee et al., 2025). Their concern is that a claim labelled "supported" may not be interrogated for how it is supported. They situate this against arguments that AI shifts knowledge work from material production to critical integration of AI output (Sarkar, 2023), and against identified metacognitive usability challenges — monitoring what the AI did, evaluating its output, and deciding how much to rely on it (Tankelevitch et al., 2024).

• Existing benchmark work goes beyond binary, but in a different direction. The paper notes that prior research has used richer labels mainly to characterise errors rather than types of positive support. Examples cited: the FRANK benchmark (Pagnoni et al., 2021), which identifies a typology of seven errors such as relation error, entity error, out-of-article error, and grammatical errors; the LibreEval dataset (Arize AI, [n.d.]), which identifies six hallucination subtypes; FActScore (Min et al., 2023), which uses a ternary top-level categorisation (supported, not supported, irrelevant) while noting finer-grained types in qualitative analysis; and AttrScore (Yue et al., 2023), which uses a ternary classification of attributable, extrapolatory, or contradictory.

• Adjacent HCI systems target adjacent but different problems. The authors distinguish their proposal from "warn-verify-audit" systems such as InkSync (Laban et al., 2024) and GenAudit (Krishna et al., 2025), which help users avoid factual errors rather than evaluate the relationship of supported claims to sources; from Traceable Texts (Kambhamettu et al., 2025a) and datatales (Sultanum and Srinivasan, 2023), which link generated phrases to source phrases; and note the closest precedent, "attribution gradients" (Kambhamettu et al., 2025b), which decomposes statements into claims and classifies each evidence source under a 2×2 framework (first-degree vs. second-degree, support vs. contradiction). They also cite co-audit design principles (Gordon et al., 2024), interactive task-decomposition interfaces exposing editable assumptions and execution plans (Kazemitabaar et al., 2024), Trailblazer for understanding agent exploration traces over a codebase (Yan et al., 2025), and the hierarchical exploration interfaces Qlarify (Fok et al., 2024) and TreeReader (Zhang et al., 2025).

• No empirical validation is reported. The paper reports no annotation study results, no inter-annotator agreement figures, no benchmark scores, and no interface evaluation. The four-system probe is presented illustratively, and the paper does not name the four commercial systems. Everything beyond the illustrative examples is proposed future work.

Methodology in Plain English

This is a position and design paper, so the "methodology" is a proposed research programme rather than an executed study. The authors first argue through worked examples that binary groundedness is insufficient. They then propose a three-part plan.

First, taxonomy construction as an iterative design process. A structured literature review across linguistics, philosophy of language, and argumentation theory would produce a longlist of candidate support relations and boundary cases. That longlist would be collapsed into a minimal working taxonomy guided by two pragmatic criteria: discriminability (can trained annotators reliably distinguish the categories?) and actionability (does the distinction matter for downstream uses such as provenance interfaces?). Expert annotators familiar with linguistic theories of support, plus non-expert readers, would iteratively review the taxonomy.

Second, an annotation specification and human validation study. The authors propose full definitions, canonical examples, counterexamples, and decision rules, written for use by both human annotators and potentially LLMs-as-judge so that annotation can scale to larger datasets and eventually online annotation. Human reliability and construct validation would be established through a human annotation study on a stratified sample of statement–source pairs, quantifying inter-annotator agreement and revealing systematic confusions between relation types so the taxonomy can be refined — ideally yielding a "gold standard" ground truth dataset.

Third, a benchmark. The plan is to construct a benchmark of statement–source pairs by enriching existing groundedness and hallucination corpora (HaluEval, HaDes, FactCHD, RAGTruth, DiaHalu, Hallulens are named as potential starting points, with a review given by Kazlaris et al., 2025) with support-relation labels. Frontier models would then be benchmarked on this annotation task by measuring their ability to match human annotations. The paper also notes that claim-spans might be identified at generation time using techniques such as symbolically grounded generation (Hennigen et al., 2024) or citation-enabled LLMs (reviewed by Gao et al., 2023), and references Claimify (Metropolitansky and Larson, 2025a) and VeriTrail (Metropolitansky and Larson, 2025b) as relevant claim-extraction and verification pipelines.

Why This Matters

Impact on research. The paper argues that current groundedness and hallucination evaluations inherit a binary framing that obscures syntactic and interpretive reformulation, and that this limits both benchmarking and interface design. A shared taxonomy of support relations could give benchmark builders a richer label space, give interface designers a vocabulary for provenance displays, and connect HCI work on augmented reading with argumentation theory, pragmatics, and scholarly-discourse representation. The authors also frame the taxonomy as a resource for developing metacognitive tools that help readers monitor, evaluate, and appropriately calibrate reliance on AI output.

Real-world applications. The paper's own description of the envisioned interface — a "relation-aware" citation interface that labels citations as PARAPHRASE, DEDUCED FROM, DIRECT QUOTE, or DEDUCED WITH ASSUMPTION (with an example assumption shown in the tooltip) and colours the corresponding text — suggests applications such as:

• Reader-facing provenance interfaces for AI-generated answers over documents, where citations convey the type of relationship rather than only the source file name.
• Document-grounded question answering and summarisation tools used in knowledge work, where readers need to decide which claims warrant closer scrutiny.
• Domain-specific analytical work where permissible interpretive moves, expected compression, and agreed evidential standards vary — the paper names legal work, medical work, and scientific research as examples.
• Fact-checking and claim-verification workflows that decompose answers into atomic claims and relate each to evidence (the territory of Traceable Texts, facts&evidence, attribution gradients, and Claimify/VeriTrail).

Industry relevance. The authors are affiliated with Microsoft Research, and the motivating examples are commercial systems' responses. The proposal speaks directly to the design of citation and groundedness features in deployed generative AI products, and to the evaluation practices those products use — including LLM-as-judge pipelines, where the paper cautions that model behaviour and evaluation can depend strongly on task framing and prompts (citing Murugadoss et al., 2025 for the finding that even highly detailed rubric instructions yield only marginal improvements in LLM-as-judge alignment with human judgements, and that simpler measures such as perplexity can sometimes perform comparably), and that LLM-generated "explanations" of their own reasoning have been shown to be unreliable because they do not reflect the model's actual mechanism (Sarkar, 2024b).

Future Directions

1. What is the smallest useful taxonomy? The authors describe a real tension between theoretical justification and operational usability at scale. Many candidate relations depend on background knowledge and context, and relations involving support contingent on ancillary assumptions can proliferate rapidly, since almost any inference can be described as requiring a tacit premise. The open design question they pose is what the smallest set of support relations is that remains meaningful for annotators and readers, informed by but not "overfitting" to philosophically motivated distinctions.

2. What are the units of analysis, and how does claim extraction relate to support labelling? It is rarely obvious what counts as a single statement, how to segment complex sentences, or when to decompose an answer into smaller atomic propositions. The paper notes that atomic decomposition can make verification tractable but introduces a consequential modelling choice about granularity and about what contextual information must remain attached for the unit to stay faithful and interpretable, raising a question about the division of labour between claim extraction and support labelling.

3. How should support be represented when evidence is distributed or internally in tension? Many answers rest on multiple non-contiguous excerpts, sometimes excerpts that pull in different directions, and real knowledge work is rarely partitioned into isolated documents — what matters is often a cloud of concepts and discussions across conversations and artefacts. The authors ask whether the relation should be defined between S and a single excerpt, between S and a set of excerpts, or between S and a structured evidential object that can represent corroboration and tension within D and across D₁,…,Dₙ, and how to treat internal inconsistency (as a distinct relation, a meta-property of the evidence set, or discursively within the generated statement).

4. How does the taxonomy transfer across tasks and domains, and what are benchmarking best practices? Support relations may manifest differently across summarisation, document-grounded question answering, and more analytical or evaluative queries, and across knowledge-work domains such as legal, medical, and scientific research, because permissible interpretive moves, expected compression, and available evidential standards vary. The authors also flag prompt sensitivity and task transfer as issues to make explicit in benchmarking protocols, and invite interdisciplinary collaboration on these questions.

Target Audience

This paper benefits most from HCI and NLP researchers working on provenance, attribution, faithfulness, and groundedness evaluation; designers and engineers building citation and source-linking features for generative AI products; benchmark and annotation-methodology researchers interested in richer label schemes beyond supported/unsupported; and scholars in argumentation theory, pragmatics, and scholarly-discourse representation who are interested in how their frameworks might be operationalised. Because it is a position paper written for a CHI 2026 workshop, it is also well suited to readers who want a compact map of adjacent work in augmented reading (Semantic Reader, CiteRead, Traceable Texts, attribution gradients, Trailblazer, Qlarify, TreeReader) and in hallucination and groundedness benchmarks (HaluEval, HaDes, FactCHD, RAGTruth, DiaHalu, Hallulens, FRANK, LibreEval, FActScore, AttrScore) rather than a finished technical result.

Authors’ abstract

Generative AI tools often answer questions using source documents, e.g., through retrieval augmented generation. Current groundedness and hallucination evaluations largely frame the relationship between an answer and its sources as binary (the answer is either supported or unsupported). However, this obscures both the syntactic moves (e.g., direct quotation vs. paraphrase) and the interpretive moves (e.g., induction vs. deduction) performed when models reformulate evidence into an answer. This limits both benchmarking and user-facing provenance interfaces. We propose the development of a reader-centred taxonomy of grounding as a set of support relations between generated statements and source documents. We explain how this might be synthesised from prior research in linguistics and philosophy of language, and evaluated through a benchmark and human annotation protocol. Such a framework would enable interfaces that communicate not just whether a claim is grounded, but how.