Whose Facts Win? LLM Source Preferences under Knowledge Conflicts

AI summary

Overview

Research area: Natural Language Processing — specifically large language models (LLMs), retrieval-augmented generation (RAG), knowledge conflicts, and the interdisciplinary study of source credibility.

Technical level: Advanced. The paper relies on forced-choice probability metrics, bootstrap significance testing, rank-aggregation methods, and a KL-divergence-based knowledge distillation setup.

Scope: The paper introduces a synthetic framework for measuring how the source of retrieved information shapes how 13 open-weight LLMs resolve conflicting information, and proposes a fine-tuning method to stop repeated information from overriding those source preferences.

What This Paper Is About

When LLMs are used in RAG pipelines, they often receive conflicting information from different retrieved documents, and prior work has studied what makes a model pick one piece of evidence over another (relevance, frequency, agreement with parametric knowledge, whether the text was LLM-generated). What no prior study had examined is whose information wins — that is, how the identity and type of the source (a government agency, a newspaper, a social media user, a private person) affects conflict resolution.

The authors build a controlled, fully synthetic testbed to isolate source preferences from real-world priors, and then test whether those preferences hold up when sources are repeated or form a majority.

Key Contributions

1. A new evaluation framework for source preferences under knowledge conflicts. Grounded in credibility research from communication, psychology, media studies, and human-computer interaction, the framework uses synthetic sources and sources' features to isolate preferences for source types without inheriting biases attached to specific real-world outlets.

2. A tightly controlled evaluation of 13 open-weight LLMs, showing that sources and their features significantly affect conflict resolution and that models follow a highly consistent credibility hierarchy.

3. A demonstration that repetition flips source preferences. The paper disentangles majority bias from repetition bias and shows that simply repeating information from a low-credibility source can reverse a model's preference — a vulnerability to repeated disinformation.

4. A fine-tuning-based mitigation method (teacher-student distillation with LoRA) that, combined with credibility-aware prompting, reduces repetition bias by up to 79.2% while retaining at least 72.5% of original source preferences, including for source types never seen during training. All data and code are released.

Main Findings

• Institutional sources win by default. Compared to a "No source available" baseline, all models prefer attributed, corroborated information. When both conflicting tables carry sources of different types, all models show strictly transitive preferences. Inter-model rankings are highly consistent (average Kendall's W of 0.74), and 11 of 13 models prefer both institutional source types over both individual source types. Using the single transferable vote algorithm with a Droop quota, the authors induce a representative overall hierarchy of government > newspaper > person, social media.

• Position bias makes the estimates conservative. Because every data instance is shown in both orders, models must overcome position bias to display any source preference. Position bias is negatively correlated with source preferences (-0.4 Spearman's ρ), meaning the measured source preferences could be even stronger.

• Popularity matters, sociodemographics barely do. All models tend to resolve conflicts in favor of higher source popularity (newspaper circulation, social media follower counts). By contrast, sociodemographic factors have only a weak effect: most models slightly prefer regional sources, sources with academic titles, women, and older people, while username style produces mixed results.

• Prompted credibility judgments diverge from behavior. Across 15 source contrasts × 13 models = 195 cases, prompting directly for credibility judgments mostly (139 out of 195 cases) elicits significantly stronger preferences in the same direction as behavioral evaluation, but models flip direction in 38 cases. Inter-model agreement rises from a Kendall's W of 0.59 to 0.77 under prompting, meaning prompting makes models more uniform but also drives some away from how they actually behave in conflict resolution.

• Models rarely abstain. Given a third option to answer "Can't be answered," most models abstain only on a minority of inputs, whether or not source information is present. Exceptions include Qwen2.5 32B (59.4% abstention without sources, 45.8% with sources) and Qwen2.5 14B (46.2% and 55.9%); OLMo-2 7B and OLMo-2 32B abstain on only 0.2% of inputs. Adding attributed sources changes abstention only marginally. Even in this three-option setting, the hierarchy remains fully transitive for all models, and 11 of 13 models reproduce the same hierarchy as in the forced-choice setup.

• Repetition, not just majority, flips preferences. When two identical tables from two different social media sources oppose one government table (2-Table Majority), all models prefer the social media sources, with an average source-preference gap of 33.90. When the same agreement is merged into a single table header (1-Table Majority), models stick with government, with an average gap of only 6.17. When the same source is repeated twice (Repetition setting), preferences again flip, with an average gap of 30.04 — even though no new source is provided and therefore no true majority exists. Repetition bias also persists for repeated unattributed information, and is even stronger when neither side has a source, indicating source credibility still plays a role but takes a back seat to repetition.

• Prompting alone is insufficient. Instructing models to identify supporting sources, assess their credibility, and ignore simple repetition weakens repetition bias but does not restore consistency with the original source hierarchy.

• Fine-tuning works. Using a teacher-student setup with LoRA on Gemma-3-4B, combined with a credibility-aware prompt, the gap between government and "no source" drops by 79.2%, from 45.9 to 9.55, while retaining 72.5% of the original preference (29.6 to 21.5). For government versus social media conflicts, repetition bias falls by 78.3%, from 33.2 to 7.2, and the preference for government slightly increases from 3.4 to 9.1. Results also make 1-Table and 2-Table majority settings behave more similarly, and they generalize to government and social media source types not seen during training.

Methodology in Plain English

Building conflicts that have no correct answer. The authors start from NeoQA, a dataset of fictional entities across seven types (art, building, event, location, organization, person, product) described by 38 attributes such as date-of-birth or headquarters. This data obeys real-world principles, units, and calendars, and was already validated with automatic and human checks. From 373 NeoQA entities, the authors create 1,903 unique counterfactually-perturbed attribute values, yielding 7,440 conflict pairs. Each pair consists of an original value and an equally plausible alternative differing in exactly one attribute. Numerical attributes are rescaled automatically by up to ±20% (dates by ±30 years within 1850–2025, or ±365 days for exact dates); categorical attributes with a small value set are sampled from LLM-generated, manually verified values; attributes with many possible values, like profession, are generated with Qwen2.5-72B. One author manually verified all alternatives, correcting formats and removing implausible cases, and proper nouns with English Wikipedia articles were removed. No variations were created for name, gender, and spouse.

Inventing sources instead of using real ones. Four source types are synthesized. Newspaper names are built from 150 of the most frequent 2-, 3-, and 4-grams extracted from U.S. newspaper names on Media Bias/Fact Check after masking location names with SpaCy; after deduplication, 59 templates such as "The {LOC} Herald" remain, filled with fictional NeoQA locations. Government sources use Qwen2.5-72B-generated agency templates per entity type. Social media users mimic Reddit's username suggestion by combining the @ symbol with WordNet adjectives and nouns plus four digits. Person sources sample the 200 most frequent first and last names from the U.S. Census Bureau and Social Security Agency between 1945 and 2007, with names having Wikipedia pages excluded.

Forced-choice prompting and a probability-based metric. Each input pairs an instruction, a context, a question, and answer options A and B. Conflicting facts are presented as Markdown tables to eliminate text-style effects. The model is queried for the deterministic probabilities of the answer tokens rather than its generations. The source preference metric is the difference between the normalized probability of answer A when source x and source y are attributed to the tables, and the normalized probability of A when no sources are attached. A positive value means source x boosts option A more than source y boosts option B. Averages are taken over the dataset, and every instance is presented in both orders to control for position bias. Significance is assessed with a nonparametric bootstrap test (n = 10,000, α = 0.05, Holm-Bonferroni correction), and only results significant for at least 10 of 13 models are reported.

Models. The 13 instruction-tuned, open-weight, decoder-only models are Qwen2.5 7B, 14B, 32B, and 72B; OLMo-2 7B, 13B, and 32B; Llama-3.2 3B, Llama-3.1 8B, and Llama-3.1 70B; and Gemma-3 4B, 12B, and 27B.

Mitigation. The fine-tuning method treats the frozen base model as a teacher and a LoRA-adapted copy as the student. Training uses 1,500 conflict pairs from 12 seed entities with newspaper and person sources, each example being an aligned pair of prompts with and without a repeated table. The loss is a weighted combination (λ = 0.2) of two Kullback–Leibler divergences: one constraining the student to mimic the teacher without repetition, one penalizing divergence under repetition. After LoRA fine-tuning, the model is trained for 200 further steps on just the first loss term to retain original preferences. Evaluation uses all conflict pairs from the remaining 361 entities combined with the previously unseen government and social media sources. Reported results are averages over 7 seeds with randomly generated held-out training entities.

Why This Matters

For research. The paper opens a new axis in the knowledge-conflict literature: prior work asked what evidence models find convincing, while this work asks whose. It also connects LLM behavior to decades of credibility research and disentangles repetition bias from majority (bandwagon) bias, showing that repetition alone can flip preferences with no majority present.

Real-world applications:

• RAG-based assistants and search: systems that retrieve from mixed corpora (official documents, news, forums, user posts) will silently weight results by source type in ways designers may not intend.
• Fact-checking and news aggregation: understanding that repeated low-credibility claims can override official ones is directly relevant to detecting coordinated disinformation that recycles the same false claim.
• Enterprise search and internal knowledge bases: when internal policies, wikis, and informal threads conflict, models will apply their own hierarchy rather than the organization's actual authority structure.
• Agentic and tool-using systems: agents that read web pages can be manipulated by simply repeating content, which the mitigation method is designed to counter.

Industry relevance. Model providers and RAG vendors get a concrete metric and dataset for auditing source sensitivity, plus a training recipe that reduces a measurable adversarial vulnerability. The released data and code lower the barrier to running similar audits on new models.

Future Directions

• Beyond English and U.S. conventions. The authors note the setup is English-only and uses U.S. naming and newspaper conventions, and that credibility judgments differ across cultures, so the hierarchy may not transfer to other languages and cultures.
• Beyond forced-choice, non-reasoning evaluation. The paper uses forced-choice MCQA with no step-by-step reasoning and no generative answers, and note that other evaluation strategies could produce different results.
• More realistic conflicts and content. The conflicts are equally plausible, tabular, and sentiment-free; real-world cases often pit a priori plausible claims against implausible ones and involve sources with genuine epistemic authority, plus style and topic effects.
• Generalizing the mitigation. The authors state that their source-aware fine-tuning may not generalize to different knowledge conflicts, contexts, and tasks in realistic RAG settings, and that they do not know how it scales.

Target Audience

Researchers and engineers working on retrieval-augmented generation, hallucination and knowledge-conflict handling, and LLM trustworthiness; evaluation and safety teams at model providers who need to audit how models weight evidence; and interdisciplinary readers in media studies, communication, or information science who study credibility and want to see how those concepts behave in LLMs. Readers need comfort with probability-based evaluation and rank-aggregation statistics to follow the results in detail.

Authors’ abstract

As large language models (LLMs) are more frequently used in retrieval-augmented generation pipelines, it is increasingly relevant to study their behavior under knowledge conflicts. Thus far, the role of the source of the retrieved information has gone unexamined. We address this gap with a novel framework to investigate how source preferences affect LLM resolution of inter-context knowledge conflicts in English, motivated by interdisciplinary research on credibility. By using synthetic sources, we study preferences for different types of sources without inheriting the biases of specific real-world sources. With a comprehensive, tightly-controlled evaluation of 13 open-weight LLMs, we find that LLMs prefer institutionally-corroborated information (e.g., government or newspaper sources) over information from people and social media. However, these source preferences can be reversed by simply repeating information from less credible sources. To mitigate repetition effects and maintain consistent preferences, we propose a novel method that reduces repetition bias by up to 79.2%, while also maintaining at least 72.5% of original preferences. We release all data and code to encourage future work on credibility and source preferences in knowledge-intensive NLP.