mlsanityData to 7 Oct 2026

ClashEval: Quantifying the tug-of-war between an LLM's internal prior and external evidence

arXiv 2404.10198 ↗First posted 16 Apr 2024cs.CL cs.AIcited by 109

On the map: Retrieval-augmented generation → RAG robustness and adaptive retrieval

Abstract (arXiv)

Retrieval augmented generation (RAG) is frequently used to mitigate hallucinations and provide up-to-date knowledge for large language models (LLMs). However, given that document retrieval is an imprecise task and sometimes results in erroneous or even harmful content being presented in context, this raises the question of how LLMs handle retrieved information: If the provided content is incorrect, does the model know to ignore it, or does it recapitulate the error? Conversely, when the model's initial response is incorrect, does it always know to use the retrieved information to correct itself, or does it insist on its wrong prior response? To answer this, we curate a dataset of over 1200 questions across six domains (e.g., drug dosages, Olympic records, locations) along with content relevant to answering each question. We further apply precise perturbations to the answers in the content that range from subtle to blatant errors. We benchmark six top-performing LLMs, including GPT-4o, on this dataset and find that LLMs are susceptible to adopting incorrect retrieved content, overriding their own correct prior knowledge over 60% of the time. However, the more unrealistic the retrieved content is (i.e. more deviated from truth), the less likely the model is to adopt it. Also, the less confident a model is in its initial response (via measuring token probabilities), the more likely it is to adopt the information in the retrieved content. We exploit this finding and demonstrate simple methods for improving model accuracy where there is conflicting retrieved content. Our results highlight a difficult task and benchmark for LLMs -- namely, their ability to correctly discern when it is wrong in light of correct retrieved content and to reject cases when the provided content is incorrect.

Key facts. Cited by 109 papers in the map (data through 7 Oct 2026). Placed in RAG robustness and adaptive retrieval, one of 15 directions in Retrieval-augmented generation. Builds on 20 of its 28 references that are in the map. First posted on arXiv 16 Apr 2024.

Builds on

20 of 28 references.

Closest ancestors · 2

Same direction, newest first

  1. 2024RetrievalQA: Assessing Adaptive Retrieval-Augmented Generation for Short-form Open-Domain Question Answering26 Feb
  2. 2023Benchmarking Large Language Models in Retrieval-Augmented Generation4 Sep

Other references · 18

  1. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,160
  2. 2022Survey of Hallucination in Natural Language Generationcited by 2,056
  3. 2023DetectGPT: Zero-Shot Machine-Generated Text Detection using Probability Curvaturecited by 705
  4. 2021Retrieval Augmentation Reduces Hallucination in Conversationcited by 691
  5. 2023Ragas: Automated Evaluation of Retrieval Augmented Generationcited by 408
  6. 2021Entity-Based Knowledge Conflicts in Question Answeringcited by 349
  7. 2023Adaptive Chameleon or Stubborn Sloth: Revealing the Behavior of Large Language Models in Knowledge Conflictscited by 333
  8. 2024TrustLLM: Trustworthiness in Large Language Modelscited by 280
  9. 2023ARES: An Automated Evaluation Framework for Retrieval-Augmented Generation Systemscited by 259
  10. 2023Challenges and Applications of Large Language Modelscited by 251
  11. 2020Generation-Augmented Retrieval for Open-domain Question Answeringcited by 186
  12. 2023Med-HALT: Medical Domain Hallucination Test for Large Language Modelscited by 156
6 more
  1. 2024Same Task, More Tokens: the Impact of Input Length on the Reasoning Performance of Large Language Modelscited by 150
  2. 2023Ever: Mitigating Hallucination in Large Language Models through Real-Time Verification and Rectificationcited by 38
  3. 2023Creating Trustworthy LLMs: Dealing with Hallucinations in Healthcare AIcited by 33
  4. 2023Evaluation of GPT-3.5 and GPT-4 for supporting real-world information needs in healthcare deliverycited by 30
  5. 2024RAGged Edges: The Double-Edged Sword of Retrieval-Augmented Chatbotscited by 11
  6. 2023RaLLe: A Framework for Developing and Evaluating Retrieval-Augmented Large Language Modelscited by 4

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 22; 2025: 43; 2026: 44.

109 papers.

Sort:

By area:

  1. 2025Words or Vision: Do Vision-Language Models Have Blind Faith in Text?cited by 86
  2. 2024FaithEval: Can Your Language Model Stay Faithful to Context, Even If "The Moon is Made of Marshmallows"cited by 85
  3. 2024Rationale-Guided Retrieval Augmented Generation for Medical Question Answeringcited by 41
  4. 2024RAG-DDR: Optimizing Retrieval-Augmented Generation Using Differentiable Data Rewardscited by 33
  5. 2024Beyond Text: Optimizing RAG with Multimodal Inputs for Industrial Applicationscited by 31
  6. 2025SafeRAG: Benchmarking Security in Retrieval-Augmented Generation of Large Language Modelcited by 31
  7. 2024To Trust or Not to Trust? Enhancing Large Language Models' Situated Faithfulness to External Contextscited by 25
  8. 2025RAG LLMs are Not Safer: A Safety Analysis of Retrieval-Augmented Generation for Large Language Modelscited by 23
  9. 2025Enhancing Retrieval-Augmented Generation: A Study of Best Practicescited by 22
  10. 2024Grounding and Evaluation for Large Language Models: Practical Challenges and Lessons Learned (Survey)cited by 20
  11. 2024KnowPO: Knowledge-aware Preference Optimization for Controllable Knowledge Selection in Retrieval-Augmented Language Modelscited by 20
  12. 2024Teaching Models to Balance Resisting and Accepting Persuasioncited by 19
38 more
  1. 2024Compressing Long Context for Enhancing RAG with AMR-based Concept Distillationcited by 16
  2. 2024On the Intrinsic Self-Correction Capability of LLMs: Uncertainty and Latent Conceptcited by 16
  3. 2025MEBench: Benchmarking Large Language Models for Cross-Document Multi-Entity Question Answeringcited by 15
  4. 2026Does RAG Know When Retrieval Is Wrong? Diagnosing Context Compliance under Knowledge Conflictcited by 15
  5. 2024Hierarchical Retrieval-Augmented Generation Model with Rethink for Multi-hop Question Answeringcited by 14
  6. 2024Parenting: Optimizing Knowledge Selection of Retrieval-Augmented Language Models with Parameter Decoupling and Tailored Tuningcited by 14
  7. 2025Don't lie to your friends: Learning what you know from collaborative self-playcited by 14
  8. 2025Shifting from Ranking to Set Selection for Retrieval Augmented Generationcited by 13
  9. 2024FineTuneBench: How well do commercial fine-tuning APIs infuse knowledge into LLMs?cited by 12
  10. 2025Feedback Friction: LLMs Struggle to Fully Incorporate External Feedbackcited by 11
  11. 2025LLM Microscope: What Model Internals Reveal About Answer Correctness and Context Utilizationcited by 11
  12. 2024Deciphering the Interplay of Parametric and Non-parametric Memory in Retrieval-augmented Language Modelscited by 10
  13. 2025Quantifying and Improving the Robustness of Retrieval-Augmented Language Models Against Spurious Features in Grounding Datacited by 10
  14. 2025Medical large language models are easily distractedcited by 9
  15. 2025MultiFinRAG: An Optimized Multimodal Retrieval-Augmented Generation (RAG) Framework for Financial Question Answeringcited by 9
  16. 2024Know Your RAG: Dataset Taxonomy and Generation Strategies for Evaluating RAG Systemscited by 8
  17. 2025MEMERAG: A Multilingual End-to-End Meta-Evaluation Benchmark for Retrieval Augmented Generationcited by 7
  18. 2025Can Large Language Models Match the Conclusions of Systematic Reviews?cited by 7
  19. 2025NeuroGenPoisoning: Neuron-Guided Attacks on Retrieval-Augmented Generation of LLM via Genetic Optimization of External Knowledgecited by 7
  20. 2025SUGAR: Leveraging Contextual Confidence for Smarter Retrievalcited by 6
  21. 2026Epistemic Context Learning: Building Trust the Right Way in LLM-Based Multi-Agent Systemscited by 6
  22. 2026Investigating Tool-Memory Conflicts in Tool-Augmented LLMscited by 5
  23. 2026Safety and accuracy follow different scaling laws in clinical large language modelscited by 5
  24. 2024Value Alignment from Unstructured Textcited by 4
  25. 2024Familiarity-Aware Evidence Compression for Retrieval-Augmented Generationcited by 4
  26. 2025Transparentize the Internal and External Knowledge Utilization in LLMs with Trustworthy Citationcited by 4
  27. 2025ConSens: Assessing context grounding in open-book question answeringcited by 4
  28. 2025Continuously Steering LLMs Sensitivity to Contextual Knowledge with Proxy Modelscited by 4
  29. 2025SafeSearch: Automated Red-Teaming of LLM-Based Search Agentscited by 4
  30. 2026Remembering More, Risking More: Longitudinal Safety Risks in Memory-Equipped LLM Agentscited by 4
  31. 2024A Tale of Trust and Accuracy: Base vs. Instruct LLMs in RAG Systemscited by 3
  32. 2025Retrieval-Augmented Generation by Evidence Retroactivity in LLMscited by 3
  33. 2025Do Large Language Models Know Conflict? Investigating Parametric vs. Non-Parametric Knowledge of LLMs for Conflict Forecastingcited by 3
  34. 2025When to Trust Context: Self-Reflective Debates for Context Reliabilitycited by 3
  35. 2025Bridging External and Parametric Knowledge: Mitigating Hallucination of LLMs with Shared-Private Semantic Synergy in Dual-Stream Knowledgecited by 3
  36. 2025KScope: A Framework for Characterizing the Knowledge Status of Language Modelscited by 3
  37. 2025ChemOrch: Empowering LLMs with Chemical Intelligence via Synthetic Instructionscited by 3
  38. 2025How Training Data Shapes the Use of Parametric and In-Context Knowledge in Language Modelscited by 3

50 most cited of 109 All 109 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2026Evaluating RAG Reliability under Clean, Misleading, and Mixed Retrieval
  2. 2024Investigating Context-Faithfulness in Large Language Models: The Roles of Memory Strength and Evidence Style
  3. 2025LLM-Specific Utility for Retrieval-Augmented Generation
  4. 2025Evaluating the Retrieval Robustness of Large Language Models
  5. 2024Long-Context LLMs Meet RAG: Overcoming Challenges for Long Inputs in RAG
  6. 2025Worse than Zero-shot? A Fact-Checking Dataset for Evaluating the Robustness of RAG Against Misleading Retrievals
  7. 2026How Do LLMs Cite? A Mechanistic Interpretation of Attribution in Retrieval-Augmented Generation
  8. 2026On the impact of retrieved content representations in RAG Pipelines