mlsanityData to 14 Oct 2026

Whose Facts Win? LLM Source Preferences under Knowledge Conflicts

arXiv 2601.03746 ↗First posted 7 Jan 2026cs.CLcited by 12

On the map: Retrieval-augmented generation → RAG robustness and adaptive retrieval

Abstract (arXiv)

As large language models (LLMs) are more frequently used in retrieval-augmented generation pipelines, it is increasingly relevant to study their behavior under knowledge conflicts. Thus far, the role of the source of the retrieved information has gone unexamined. We address this gap with a novel framework to investigate how source preferences affect LLM resolution of inter-context knowledge conflicts in English, motivated by interdisciplinary research on credibility. By using synthetic sources, we study preferences for different types of sources without inheriting the biases of specific real-world sources. With a comprehensive, tightly-controlled evaluation of 13 open-weight LLMs, we find that LLMs prefer institutionally-corroborated information (e.g., government or newspaper sources) over information from people and social media. However, these source preferences can be reversed by simply repeating information from less credible sources. To mitigate repetition effects and maintain consistent preferences, we propose a novel method that reduces repetition bias by up to 79.2%, while also maintaining at least 72.5% of original preferences. We release all data and code to encourage future work on credibility and source preferences in knowledge-intensive NLP.

Key facts. Cited by 12 papers in the map (data through 14 Oct 2026). Placed in RAG robustness and adaptive retrieval, one of 15 directions in Retrieval-augmented generation. Builds on 36 of its 91 references that are in the map. First posted on arXiv 7 Jan 2026.

Builds on

36 of 91 references.

Closest ancestors · 5

Same direction, newest first

  1. 2025NeoQA: Evidence-based Question Answering with Generated News Events9 May
  2. 2025Transparentize the Internal and External Knowledge Utilization in LLMs with Trustworthy Citation21 Apr
  3. 2024Not All Contexts Are Equal: Teaching LLMs Credibility-aware Generation10 Apr
  4. 2024Tug-of-War Between Knowledge: Exploring and Resolving Knowledge Conflicts in Retrieval-Augmented Language Models22 Feb
  5. 2023Why So Gullible? Enhancing the Robustness of Retrieval-Augmented Models against Counterfactual Noise2 May

Other references · 31

  1. 2022Training language models to follow instructions with human feedbackcited by 15,971
  2. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,297
  3. 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,626
  4. 2023Quantifying Language Models' Sensitivity to Spurious Features in Prompt Design or: How I learned to start worrying about prompt formattingcited by 817
  5. 2024Justice or Prejudice? Quantifying Biases in LLM-as-a-Judgecited by 368
  6. 2021Entity-Based Knowledge Conflicts in Question Answeringcited by 356
  7. 2023Adaptive Chameleon or Stubborn Sloth: Revealing the Behavior of Large Language Models in Knowledge Conflictscited by 341
  8. 2024Humans or LLMs as the Judge? A Study on Judgement Biasescited by 323
  9. 2023State of What Art? A Call for Multi-Prompt LLM Evaluationcited by 310
  10. 2024Knowledge Conflicts for LLMs: A Surveycited by 283
  11. 2020Intrinsic Bias Metrics Do Not Correlate with Application Biascited by 148
  12. 2022Rich Knowledge Sources Bring Complex Knowledge Conflicts: Recalibrating Models to Reflect Conflicting Evidencecited by 144
19 more
  1. 2024What Evidence Do Language Models Find Convincing?cited by 75
  2. 2024ConflictBank: A Benchmark for Evaluating the Influence of Knowledge Conflicts in LLMcited by 73
  3. 2024Do LLMs Know When to NOT Answer? Investigating Abstention Abilities of Large Language Modelscited by 57
  4. 2025Mind the Value-Action Gap: Do LLMs Act in Alignment with Their Values?cited by 51
  5. 2024Blinded by Generated Contexts: How Language Models Merge Generated and Retrieved Contexts When Knowledge Conflicts?cited by 48
  6. 2021Attacking Open-domain Question Answering by Injecting Misinformationcited by 41
  7. 2023Two Failures of Self-Consistency in the Multi-Step Reasoning of LLMscited by 36
  8. 2024DYNAMICQA: Tracing Internal Knowledge Conflicts in Language Modelscited by 34
  9. 2024A Comprehensive Evaluation of Cognitive Biases in LLMscited by 34
  10. 2025Assessing Judging Bias in Large Reasoning Models: An Empirical Studycited by 30
  11. 2024Prompt Perturbation Consistency Learning for Robust Language Modelscited by 26
  12. 2023Accuracy and Political Bias of News Source Credibility Ratings by Large Language Modelscited by 25
  13. 2020Early Detection of Fake News by Utilizing the Credibility of News, Publishers, and Users Based on Weakly Supervised Learningcited by 23
  14. 2024Open Domain Question Answering with Conflicting Contextscited by 18
  15. 2024Robust Pronoun Fidelity with English LLMs: Are they Reasoning, Repeating, or Just Biased?cited by 17
  16. 2024Adaptive Question Answering: Enhancing Language Model Proficiency for Addressing Knowledge Conflicts with Source Citationscited by 15
  17. 2025Agree to Disagree? A Meta-Evaluation of LLM Misgenderingcited by 7
  18. 2021Determining the Credibility of Science Communicationcited by 4
  19. 2025Format as a Prior: Quantifying and Analyzing Bias in LLMs for Heterogeneous Datacited by 4

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 12.

12 papers.

  1. 2026Most LLM Conformity Needs No Speaker: Measuring the Speaker-Free Floor in Peer-Pressure Benchmarkscited by 5
  2. 2026When Personal Memory Has No Single Answer: Evaluating LLM Agents under Irreducible Conflictcited by 1
  3. 2026When LLMs Sit Above Diagnostic Tools: Unrealized Complementarity in Industrial Fault Diagnosiscited by 1
  4. 2026Language Bias under Conflicting Information in Multilingual LLMscited by 0
  5. 2026Authority, Truth, and Citation Bias: A Large-Scale Multi-Domain Benchmark for Studying Epistemic Susceptibility in Large Language Modelscited by 0
  6. 2026Shortcut Before Circuit: Document Statistics Time In-Context Conflict Resolutioncited by 0
  7. 2026Record Grouping Controls Evidence Weight in Language Modelscited by 0
  8. 2026CITECHOICE: A Causal Audit of How Document Presentation Redistributes Citation Credit in Agentic Searchcited by 0
  9. 2026Where's Waldo? Query-language Preference under Cross-lingual Knowledge Disparitiescited by 0
  10. 2026Source Preference in the Wild: How LLM Agents Favor Items by Source, and How to Reduce Itcited by 0
  11. 2026Copies or Sources? Measuring How LLM Aggregators Count Restated Evidence in Multi-Agent Systemscited by 0
  12. 2026Provenance-Aware Belief Updating in Human-AI Feedback Loopscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026Which Knowledge Should LLMs Trust? Uncovering a Generalizable Steering Direction Across Knowledge Conflicts
  2. 2026In Agents We Trust, but Who Do Agents Trust? Latent Source Preferences Steer LLM Generations
  3. 2026Exploring Knowledge Conflicts for Faithful LLM Reasoning: Benchmark and Method
  4. 2025Resolving Conflicting Evidence in Automated Fact-Checking: A Study on Retrieval-Augmented LLMs
  5. 2025DRAGged into Conflicts: Detecting and Addressing Conflicting Sources in Search-Augmented LLMs
  6. 2025Accommodate Knowledge Conflicts in Retrieval-augmented LLMs: Towards Robust Response Generation in the Wild
  7. 2024WikiContradict: A Benchmark for Evaluating LLMs on Real-World Knowledge Conflicts from Wikipedia
  8. 2023Survey on Factuality in Large Language Models: Knowledge, Retrieval and Domain-Specificity