mlsanityData to 14 Oct 2026

DRAGged into Conflicts: Detecting and Addressing Conflicting Sources in Search-Augmented LLMs

arXiv 2506.08500 ↗First posted 10 Jun 2025cs.CL cs.AI cs.LGcited by 25

On the map: Retrieval-augmented generation → RAG robustness and adaptive retrieval

Abstract (arXiv)

Retrieval Augmented Generation (RAG) is a commonly used approach for enhancing large language models (LLMs) with relevant and up-to-date information. However, the retrieved sources can often contain conflicting information and it remains unclear how models should address such discrepancies. In this work, we first propose a novel taxonomy of knowledge conflict types in RAG, along with the desired model behavior for each type. We then introduce CONFLICTS, a high-quality benchmark with expert annotations of conflict types in a realistic RAG setting. CONFLICTS is the first benchmark that enables tracking progress on how models address a wide range of knowledge conflicts. We conduct extensive experiments on this benchmark, showing that LLMs often struggle to appropriately resolve conflicts between sources. While prompting LLMs to explicitly reason about the potential conflict in the retrieved documents significantly improves the quality and appropriateness of their responses, substantial room for improvement in future research remains.

Key facts. Cited by 25 papers in the map (data through 14 Oct 2026). Placed in RAG robustness and adaptive retrieval, one of 15 directions in Retrieval-augmented generation. Builds on 45 of its 53 references that are in the map. First posted on arXiv 10 Jun 2025.

Builds on

45 of 53 references.

Closest ancestors · 6

Same direction, newest first

  1. 2025Retrieval-Augmented Generation with Conflicting Evidence17 Apr
  2. 2024FaithEval: Can Your Language Model Stay Faithful to Context, Even If "The Moon is Made of Marshmallows"30 Sep
  3. 2024WikiContradict: A Benchmark for Evaluating LLMs on Real-World Knowledge Conflicts from Wikipedia19 Jun
  4. 2024Studying Large Language Model Behaviors Under Context-Memory Conflicts With Real Documents24 Apr
  5. 2024MiniCheck: Efficient Fact-Checking of LLMs on Grounding Documents16 Apr
  6. 2024Enhancing LLM Factual Accuracy with RAG to Counter Hallucinations: A Case Study on Domain-Specific Queries in Private Knowledge-Bases15 Mar

Other references · 39

  1. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,297
  2. 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,626
  3. 2023Lost in the Middle: How Language Models Use Long Contextscited by 3,455
  4. 2023G-Eval: NLG Evaluation using GPT-4 with Better Human Alignmentcited by 2,276
  5. 2023Retrieval-Augmented Generation for Large Language Models: A Surveycited by 2,212
  6. 2020REALM: Retrieval-Augmented Language Model Pre-Trainingcited by 2,141
  7. 2025Gemma 3 Technical Reportcited by 1,824
  8. 2021Improving language models by retrieving from trillions of tokenscited by 1,293
  9. 2022When Not to Trust Language Models: Investigating Effectiveness of Parametric and Non-Parametric Memoriescited by 1,128
  10. 2022Atlas: Few-shot Learning with Retrieval Augmented Language Modelscited by 952
  11. 2023Can Large Language Models Be an Alternative to Human Evaluations?cited by 734
  12. 2023In-Context Retrieval-Augmented Language Modelscited by 702
27 more
  1. 2023Enabling Large Language Models to Generate Text with Citationscited by 595
  2. 2023Large Language Models Are State-of-the-Art Evaluators of Translation Qualitycited by 420
  3. 2020AmbigQA: Answering Ambiguous Open-domain Questionscited by 390
  4. 2023Making Retrieval-Augmented Language Models Robust to Irrelevant Contextcited by 369
  5. 2021Entity-Based Knowledge Conflicts in Question Answeringcited by 356
  6. 2024Knowledge Conflicts for LLMs: A Surveycited by 283
  7. 2021Measuring Attribution in Natural Language Generation Modelscited by 256
  8. 2022RealTime QA: What's the Answer Right Now?cited by 244
  9. 2021Efficiently Teaching an Effective Dense Retriever with Balanced Topic Aware Samplingcited by 238
  10. 2022ASQA: Factoid Questions Meet Long-Form Answerscited by 234
  11. 2021SituatedQA: Incorporating Extra-Linguistic Contexts into QAcited by 145
  12. 2019Asking Clarifying Questions in Open-Domain Information-Seeking Conversationscited by 139
  13. 2024VERISCORE: Evaluating the factuality of verifiable claims in long-form text generationcited by 117
  14. 2023A Confederacy of Models: a Comprehensive Evaluation of LLMs on Creative Writingcited by 99
  15. 2023Clarify When Necessary: Resolving Ambiguity Through Interaction with LMscited by 98
  16. 2024Can Large Language Models Faithfully Express Their Intrinsic Uncertainty in Words?cited by 87
  17. 2024What Evidence Do Language Models Find Convincing?cited by 75
  18. 2024Attribute First, then Generate: Locally-attributable Grounded Text Generationcited by 54
  19. 2025The FACTS Grounding Leaderboard: Benchmarking LLMs' Ability to Ground Responses to Long-Form Inputcited by 52
  20. 2024Blinded by Generated Contexts: How Language Models Merge Generated and Retrieved Contexts When Knowledge Conflicts?cited by 48
  21. 2022Synthetic Disinformation Attacks on Automated Fact Verification Systemscited by 44
  22. 2024AmbigDocs: Reasoning across Documents on Different Entities under the Same Namecited by 21
  23. 2023Towards Better Evaluation of Instruction-Following: A Case-Study in Summarizationcited by 18
  24. 2024ECon: On the Detection and Resolution of Evidence Conflictscited by 18
  25. 2024Open Domain Question Answering with Conflicting Contextscited by 18
  26. 2024Localizing Factual Inconsistencies in Attributable Text Generationcited by 14
  27. 2024DebateQA: Evaluating Question Answering on Debatable Knowledgecited by 12

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 5; 2026: 20.

25 papers.

Sort:

By area:

  1. 2025Deep Research: A Survey of Autonomous Research Agentscited by 66
  2. 2025A Survey of LLM-based Deep Search Agents: Paradigm, Optimization, Evaluation, and Challengescited by 62
  3. 2026Agentic Environment Engineering for Large Language Models: A Survey of Environment Modeling, Synthesis, Evaluation, and Applicationcited by 8
  4. 2025A Unified Definition of Hallucination: It's The World Model, Stupid!cited by 6
  5. 2025Consensus or Conflict? Fine-Grained Evaluation of Conflicting Answers in Question-Answeringcited by 5
  6. 2026EvidenceRL: Reinforcing Evidence Consistency for Trustworthy Language Modelscited by 5
  7. 2026ConflictRAG: Detecting and Resolving Knowledge Conflicts in Retrieval Augmented Generationcited by 4
  8. 2026Addressing Corpus Knowledge Poisoning Attacks on RAG Using Sparse Attentioncited by 3
  9. 2026To Isolate or to Score? Model-Adaptive Assessment for Cost-Efficient Multi-Agent RAGcited by 2
  10. 2026When Abundance Conceals Weakness: Knowledge Conflict in Multilingual Modelscited by 1
  11. 2026CryptoAnalystBench: Failures in Multi-Tool Long-Form LLM Analysiscited by 1
  12. 2026Retrieval-Augmented Generation Must Move Beyond Factual Grounding to Represent Diverse Opinionscited by 1
13 more
  1. 2026Enhancing LLM Metacognition via Cognitive Pairwise Trainingcited by 1
  2. 2026When Personal Memory Has No Single Answer: Evaluating LLM Agents under Irreducible Conflictcited by 1
  3. 2026Towards Evolving Context Parameterization for Large Language Modelscited by 1
  4. 2025On Finding Inconsistencies in Documentscited by 0
  5. 2026DETOUR: An Interactive Benchmark for Dual-Agent Search and Reasoningcited by 0
  6. 2026DeepStress: Stress-Testing Deep Search Agentscited by 0
  7. 2026Detecting Knowledge Inconsistencies Across Text, Tables, and Knowledge Graphscited by 0
  8. 2026Large Language Models in Resolving Contextual Knowledge Conflictscited by 0
  9. 2026Stochastic Semantic Evidence Graphs: Uncertainty Propagation and Governance for Agentic AIcited by 0
  10. 2026Where's Waldo? Query-language Preference under Cross-lingual Knowledge Disparitiescited by 0
  11. 2026InvestigationWorlds: An Agentic Environment for Legal Investigationcited by 0
  12. 2026Benchmarking and Training Reward Models for Cross-Source Evidence Grounding in Search Agentscited by 0
  13. 2026Which Knowledge Should LLMs Trust? Uncovering a Generalizable Steering Direction Across Knowledge Conflictscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025MAGIC: A Multi-Hop and Graph-Based Benchmark for Inter-Context Conflicts in Retrieval-Augmented Generation
  2. 2026Exploring Knowledge Conflicts for Faithful LLM Reasoning: Benchmark and Method
  3. 2024Who's Who: Large Language Models Meet Knowledge Conflicts in Practice
  4. 2025Rethinking All Evidence: Enhancing Trustworthy Retrieval-Augmented Generation via Conflict-Driven Summarization
  5. 2025TruthfulRAG: Resolving Factual-level Conflicts in Retrieval-Augmented Generation with Knowledge Graphs
  6. 2024Astute RAG: Overcoming Imperfect Retrieval Augmentation and Knowledge Conflicts for Large Language Models
  7. 2025From Facts to Conclusions : Integrating Deductive Reasoning in Retrieval-Augmented LLMs
  8. 2025Accommodate Knowledge Conflicts in Retrieval-augmented LLMs: Towards Robust Response Generation in the Wild