It's Not What You Say, It's How You Say It: Evaluating LLM Responses to Expressions of Belief

AI summary

Overview

Research area: Natural Language Processing, specifically LLM evaluation at the intersection of formal pragmatics/semantics and knowledge-conflict behavior.

Technical level: Intermediate. The core idea is intuitive (does how you say something change whether a model believes you?), but the paper builds on linguistic theory (presupposition, evidentiality, epistemic stance, register) and uses a controlled benchmark with statistical ranking tests.

Scope: The authors introduce EoBench, a 65,778-item benchmark built from a four-dimension, 19-type typology of "expressions of belief," and use it to measure how 18 LLMs (16 after filtering) trade off in-context belief against prior parametric knowledge.

What This Paper Is About

Users constantly tell language models things, but they say them in very different ways: as flat assertions, presuppositions, hedged guesses, commands, sarcastic remarks, or appeals to authority. The paper asks whether these linguistic differences actually change whether a model abandons a fact it already knows in favor of a false claim stated in context. To answer this, the authors build a controlled benchmark that holds the underlying proposition constant and varies only how the belief is expressed.

Key Contributions

1. A linguistically grounded typology of expressions of belief (EoBs). The typology spans four dimensions — form, evidentiality, epistemic stance, and tone — covering 19 fine-grained types (7 form types, 3 evidentiality types, 2 epistemic stance types, 7 tone types). Types are drawn from prior linguistic and philosophical work (Lyons, Stalnaker, Aikhenvald, and others).

2. A programmatic generation framework and the EoBench dataset. Beliefs are abstracted as semantic tuples containing a subject, relation, object, object_pri (the factually correct answer), extra information about the object, an authority source, a belief source, a material condition, and a counterfactual condition. Each EoB type has 10 templates. From 3,462 filtered PopQA-derived tuples crossed with 19 EoB types, the authors generate 65,778 expressions of belief.

3. A systematic evaluation of 18 LLMs. Models span three families (Gemma 3, Llama 3.1/3.2, Qwen3), sizes from 1B to 30B parameters, and both base and instruction-tuned training stages, scored with a context-following rate (CFR) metric.

4. Fine-grained findings on what persuades models. The paper identifies specific EoB types that are statistically significantly more or less persuasive across models, and shows that model scale, training stage, and family each shift context-following behavior.

Main Findings

• Some EoBs persuade more than others. Averaged across all model families, sizes, and training types, the EoB types child-directed, imperative, formal, appeals to authority, suppositions, and emotional appeals ranked as significantly more context-following than a random baseline. Examples given: "Okay sweetie, so the capital of France is London." and "Remember that London is the capital of France."

• Some EoBs are markedly unconvincing. Counterfactual, belief reports, weak (epistemic stance), and interrogative EoBs ranked as significantly less context-following than a random baseline. Examples: "If Berlin weren't the capital of Germany, London would be the capital of France." and "My brother believes the capital of France is London."

• Instruction tuning reduces context-following. Across most EoB dimensions, base models were generally as susceptible — or more susceptible — to contradictory context than their instruction-tuned counterparts. A notable exception is Qwen3-8B, whose instruction-tuned variant agreed with the in-context belief 18.1% more than the base model on average.

• Dimension-level differences are informative. For Llama3-8B, Llama3-3B, and Gemma3-12B, the largest base-vs-instruct gap falls in the epistemic stance dimension. For Llama3-1B, instruction tuning cuts both ways: the base variant is much more context-following on epistemic stance but much less context-following on evidentiality.

• Bigger models are less persuaded. For Llama3 and Gemma3, CFR tends to decrease as model size increases. Gemma3 shows this clearly from 1B to 30B — the Gemma3-1B instruct model is described as almost entirely context-following. Qwen3 instruct models instead show a relatively stable trend with high CFR across sizes and dimensions, except for the evidentiality dimension.

• Dimension disparity grows with scale. The gap in CFR between different EoB dimensions appears to increase with model size for Gemma3 at all sizes, Qwen3 instruct models, and Llama3 Instruct from 3B to 8B. Llama3 shows a more mixed pattern; it is unclear whether Llama3-1B-Instruct's high disparity is anomalous, since that model is relatively poor at instruction-following on IFEval compared to similarly sized Gemma3 and Qwen3 models.

• Model family matters. Llama3 models have relatively low CFRs compared with same-size models from the other families, and lower CFR than Gemma3 and Qwen3 on most dimensions.

• Prior knowledge quality varies widely across models. The best performers on the prior-knowledge check were Qwen3-8B-Base, Gemma3-12B-Instruct, and Gemma3-27B-Instruct, all with over 30,000 correct examples. The lowest were Gemma3-1B (Base) with 4,732 examples and Qwen3-1.7B-Base with 383 examples; both were excluded from downstream analysis.

Methodology in Plain English

The authors start from PopQA, an open-domain Wikidata-based QA dataset of 14k (subject, relation, object_pri) tuples spanning 15 topics such as author, capital, and genre. They filter it in several steps: keep only facts where a subject and relation map to a single object, so that asserting an alternative genuinely contradicts the original; keep objects linked to two or more facts in PopQA so extra information is available; drop objects shorter than 2 characters, longer than 50 characters, or entirely numerical; then downsample to 3,462 triples distributed across the 15 relations proportionally to the original dataset. For each fact, they sample a different object within the same relation to create the conflicting belief.

Each semantic tuple is then run through JSON-based parameterized templates to produce one EoB per typology type, with 10 templates available per type, yielding 65,778 EoBs. All EoBs are factually incorrect, so agreeing with the context is a clear signal that the model accepted a false belief.

To score a model, the authors ask two yes–no questions: whether the original factual proposition holds and whether the conflicting proposition holds, asking in both directions to counter the tendency of models to favor "no" over "yes." A model is counted as context-following only if it answers consistently for the EoB; responses other than Yes or No are filtered out. Separately, each model is tested on the facts without the EoB to confirm it actually knows the prior answer, and models answering fewer than 10% of facts correctly are dropped — this excludes Gemma3-1B and Qwen3-1.7B-Base, leaving 16 models.

The main metric is context-following rate (CFR): the number of queries where the model agreed with the in-context EoB divided by the number of queries where it agreed with either the EoB or the original fact. This approximates P(Y = context | Y is context or memory). To identify which EoB types are consistently more persuasive, they run a permutation test (k = 1000, α = 0.05) with the Benjamini–Hochberg correction, testing whether each type's mean ranking across the 16 models differs from the mean ranking under random shuffling.

Why This Matters

The paper argues that model robustness and prompt engineering advice cannot be built on "context versus memory" alone, because the linguistic packaging of a belief is itself a strong determinant of model behavior. Whether a model should follow context is setting-dependent: a chat assistant arguably should adapt to a user's stated beliefs, while a knowledge-grounded task should resist false claims. Without a controlled measurement of how form, evidentiality, stance, and tone shift this tradeoff, evaluations of context-sensitivity are incomplete.

Real-world applications:

• Chat assistants and personalization. Understanding which phrasings a model over-trusts helps calibrate how readily an assistant adopts user-supplied premises.
• Misinformation resistance. The findings show that tones like child-directed or emotional appeals and framings like appeals to authority are particularly effective at getting models to accept false facts.
• Retrieval-augmented and knowledge-grounded systems. These pipelines rely on injected context being treated as reliable, so knowing which EoB forms are over- or under-weighted matters for grounding decisions.
• Prompt engineering and evaluation practice. The benchmark gives practitioners a controlled suite for testing robustness to belief framing rather than only to raw factual contradiction.

Industry relevance: The results suggest that instruction tuning and scaling both reduce context-following, which is directly relevant to alignment and post-training decisions for deployed assistants, and to building adversarial evaluations that surface framing-based vulnerabilities.

Future Directions

• Extend the typology. The authors state their four dimensions are not exhaustive and could be extended with additional categories or finer-grained distinctions. They also present only single-dimension assertions; cross-dimensional expressions (e.g., "Jane believes that London, the capital of France, is the coolest city in the world") are noted as significantly harder to template and analyze.
• Mechanistic and causal analysis. The authors suggest causal abstraction and mechanistic interpretability to test whether LLMs process different EoBs differently at a computational level, motivated by neuroscience evidence that humans process assertions, presuppositions, and counterfactuals via distinct neural circuits.
• Explain the Llama3 outlier. It remains unclear why Llama3 models appear less context-following than comparable models; the authors propose investigating how pretraining and post-training data influence susceptibility to different EoBs.
• Move beyond synthetic false beliefs. The limitations section notes that experiments use incorrect assertions and template-based generations, which may not capture natural language variability or true-assertion handling.

Target Audience

Researchers working on LLM evaluation, alignment, and post-training will find the benchmark and cross-model comparisons most directly useful, as will computational linguists and pragmatics researchers interested in whether linguistic distinctions like presupposition and evidentiality carry over to model behavior. Practitioners doing prompt engineering, building retrieval-augmented or knowledge-grounded systems, or designing adversarial robustness evaluations also benefit from the fine-grained catalog of which EoB types are most and least persuasive.

Authors’ abstract

Users frequently express their beliefs to large language models (LLMs). In some situations, the LLM should accept these contextual beliefs as true. In others, they should stick to their prior knowledge. Notably, users' expressions of belief (EoBs) can take linguistically diverse forms - using presuppositions, evidential and certainty markers, or varied tones - each of which may have a different persuasiveness over the LLMs. We introduce a typology to systematically evaluate how different EoBs affect whether models follow context versus prior knowledge. The typology is grounded in four linguistically motivated dimensions: form, evidentiality, epistemic stance, and tone, spanning 17 fine-grained types. By pairing these EoBs with world knowledge facts, we generate controlled EoB-query pairs that isolate the effect of linguistic variation. Using this benchmark, we evaluate 16 LLMs that differ in architecture (Llama3, Qwen3, Gemma3), scale (1B-30B parameters), and training stages (base vs instruct). We identify meaningful variations in response behavior across these axes, e.g., that bigger models and instruction models tend to be less context-following than smaller models and base models. We further identify specific EoBs that statistically significantly persuade LMs more consistently than others. Our work reveals systematic patterns in how linguistic framing affects LLM context integration, with implications for prompt engineering and model robustness.