mlsanityData to 14 Oct 2026

Who's Who: Large Language Models Meet Knowledge Conflicts in Practice

arXiv 2410.15737 ↗First posted 21 Oct 2024cs.CL cs.AI cs.IRcited by 21

On the map: Retrieval-augmented generation → RAG robustness and adaptive retrieval

Abstract (arXiv)

Retrieval-augmented generation (RAG) methods are viable solutions for addressing the static memory limits of pre-trained language models. Nevertheless, encountering conflicting sources of information within the retrieval context is an inevitable practical challenge. In such situations, the language models are recommended to transparently inform users about the conflicts rather than autonomously deciding what to present based on their inherent biases. To analyze how current large language models (LLMs) align with our recommendation, we introduce WhoQA, a public benchmark dataset to examine model's behavior in knowledge conflict situations. We induce conflicts by asking about a common property among entities having the same name, resulting in questions with up to 8 distinctive answers. WhoQA evaluation set includes 5K questions across 13 Wikidata property types and 150K Wikipedia entities. Our experiments show that despite the simplicity of WhoQA questions, knowledge conflicts significantly degrades LLMs' performance in RAG settings.

Key facts. Cited by 21 papers in the map (data through 14 Oct 2026). Placed in RAG robustness and adaptive retrieval, one of 15 directions in Retrieval-augmented generation. Builds on 17 of its 25 references that are in the map. First posted on arXiv 21 Oct 2024.

Builds on

17 of 25 references.

References · 17

  1. 2020Language Models are Few-Shot Learnerscited by 32,419
  2. 2022Training language models to follow instructions with human feedbackcited by 15,971
  3. 2024The Llama 3 Herd of Modelscited by 14,715
  4. 2023Efficient Memory Management for Large Language Model Serving with PagedAttentioncited by 5,944
  5. 2022PaLM: Scaling Language Modeling with Pathwayscited by 5,326
  6. 2021Unsupervised Dense Information Retrieval with Contrastive Learningcited by 1,216
  7. 2020AmbigQA: Answering Ambiguous Open-domain Questionscited by 390
  8. 2021Entity-Based Knowledge Conflicts in Question Answeringcited by 356
  9. 2023Adaptive Chameleon or Stubborn Sloth: Revealing the Behavior of Large Language Models in Knowledge Conflictscited by 341
  10. 2021Time-Aware Language Models as Temporal Knowledge Basescited by 296
  11. 2020ToTTo: A Controlled Table-To-Text Generation Datasetcited by 268
  12. 2021SituatedQA: Incorporating Extra-Linguistic Contexts into QAcited by 145
5 more
  1. 2022DisentQA: Disentangling Parametric and Contextual Knowledge with Counterfactual Question Answeringcited by 96
  2. 2021Time Waits for No One! Analysis and Challenges of Temporal Misalignmentcited by 91
  3. 2024Unfamiliar Finetuning Examples Control How Language Models Hallucinatecited by 87
  4. 2020Revisiting Challenges in Data-to-Text Generation with Fact Groundingcited by 27
  5. 2024Time Sensitive Knowledge Editing through Efficient Finetuningcited by 14

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 1; 2025: 10; 2026: 10.

21 papers.

Sort:

By area:

  1. 2025M+: Extending MemoryLLM with Scalable Long-Term Memorycited by 54
  2. 2026STALE: Can LLM Agents Know When Their Memories Are No Longer Valid?cited by 50
  3. 2024MRAG: A Modular Retrieval Framework for Time-Sensitive Question Answeringcited by 17
  4. 2025What Makes a Good Natural Language Prompt?cited by 7
  5. 2025A Unified Definition of Hallucination: It's The World Model, Stupid!cited by 6
  6. 2025ConSens: Assessing context grounding in open-book question answeringcited by 4
  7. 2025Format as a Prior: Quantifying and Analyzing Bias in LLMs for Heterogeneous Datacited by 4
  8. 2026Faithfulness vs. Safety: Evaluating LLM Behavior Under Counterfactual Medical Evidencecited by 3
  9. 2025FactReasoner: A Probabilistic Approach to Long-Form Factuality Assessment for Large Language Modelscited by 2
  10. 2025Conflicts in Texts: Data, Implications and Challengescited by 2
  11. 2025Harnessing the Unseen: The Hidden Influence of Intrinsic Knowledge in Long-Context Language Modelscited by 1
  12. 2026Navigating Unreliable Parametric and Contextual Knowledge: Explicit Knowledge Conflict Resolution for LLM Inferencecited by 1
9 more
  1. 2026Information Discernment in Large Language Modelscited by 1
  2. 2025ConceptCarve: Dynamic Realization of Evidencecited by 0
  3. 2025On Finding Inconsistencies in Documentscited by 0
  4. 2026ConflictScore: Identifying and Measuring How Language Models Handle Conflicting Evidencecited by 0
  5. 2026Hierarchical Graph Memory for LLM Agents with Path-level Localization and Rewritecited by 0
  6. 2026Blind Men and the Elephant: Probing the Epistemic Myopia of LLMs under Long-Tail Divergent Knowledgecited by 0
  7. 2026Identifying Temporal Features within Transcoders for Time Sensitive Factual Recallcited by 0
  8. 2026BELIEFRAG: Making Adaptive RAG State-Aware under Evolving Evidencecited by 0
  9. 2026Benchmarking and Training Reward Models for Cross-Source Evidence Grounding in Search Agentscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025DRAGged into Conflicts: Detecting and Addressing Conflicting Sources in Search-Augmented LLMs
  2. 2024WikiContradict: A Benchmark for Evaluating LLMs on Real-World Knowledge Conflicts from Wikipedia
  3. 2025TruthfulRAG: Resolving Factual-level Conflicts in Retrieval-Augmented Generation with Knowledge Graphs
  4. 2026Exploring Knowledge Conflicts for Faithful LLM Reasoning: Benchmark and Method
  5. 2025FaithfulRAG: Fact-Level Conflict Modeling for Context-Faithful Retrieval-Augmented Generation
  6. 2024Studying Large Language Model Behaviors Under Context-Memory Conflicts With Real Documents
  7. 2025MAGIC: A Multi-Hop and Graph-Based Benchmark for Inter-Context Conflicts in Retrieval-Augmented Generation
  8. 2025Rethinking All Evidence: Enhancing Trustworthy Retrieval-Augmented Generation via Conflict-Driven Summarization