# Resolving Knowledge Conflicts in Large Language Models

arXiv: https://arxiv.org/abs/2310.00935 · HTML: https://mlsanity.com/paper/2310.00935/

## Key facts

- Cited by 50 papers in the map (data through 14 Oct 2026).
- Builds on 47 of its 64 references that are in the map.
- First posted on arXiv 2 Oct 2023.

## Builds on (most cited first, 30 of 47)

- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [GPT-4 Technical Report](https://mlsanity.com/paper/2303.08774/) (2023)
- [Training language models to follow instructions with human feedback](https://mlsanity.com/paper/2203.02155/) (2022)
- [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019)
- [Chain-of-Thought Prompting Elicits Reasoning in Large Language Models](https://mlsanity.com/paper/2201.11903/) (2022)
- [Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks](https://mlsanity.com/paper/2005.11401/) (2020)
- [Self-Consistency Improves Chain of Thought Reasoning in Language Models](https://mlsanity.com/paper/2203.11171/) (2022)
- [Lost in the Middle: How Language Models Use Long Contexts](https://mlsanity.com/paper/2307.03172/) (2023)
- [Sparks of Artificial General Intelligence: Early experiments with GPT-4](https://mlsanity.com/paper/2303.12712/) (2023)
- [Pre-train, Prompt, and Predict: A Systematic Survey of Prompting Methods in Natural Language Processing](https://mlsanity.com/paper/2107.13586/) (2021)
- [REALM: Retrieval-Augmented Language Model Pre-Training](https://mlsanity.com/paper/2002.08909/) (2020)
- [Survey of Hallucination in Natural Language Generation](https://mlsanity.com/paper/2202.03629/) (2022)
- [Language Models as Knowledge Bases?](https://mlsanity.com/paper/1909.01066/) (2019)
- [Improving Factuality and Reasoning in Language Models through Multiagent Debate](https://mlsanity.com/paper/2305.14325/) (2023)
- [WebGPT: Browser-assisted question-answering with human feedback](https://mlsanity.com/paper/2112.09332/) (2021)
- [Improving language models by retrieving from trillions of tokens](https://mlsanity.com/paper/2112.04426/) (2021)
- [When Not to Trust Language Models: Investigating Effectiveness of Parametric and Non-Parametric Memories](https://mlsanity.com/paper/2212.10511/) (2022)
- [Measuring and Narrowing the Compositionality Gap in Language Models](https://mlsanity.com/paper/2210.03350/) (2022)
- [Large Language Models Can Be Easily Distracted by Irrelevant Context](https://mlsanity.com/paper/2302.00093/) (2023)
- [REPLUG: Retrieval-Augmented Black-Box Language Models](https://mlsanity.com/paper/2301.12652/) (2023)
- [Extending Context Window of Large Language Models via Positional Interpolation](https://mlsanity.com/paper/2306.15595/) (2023)
- [How Much Knowledge Can You Pack Into the Parameters of a Language Model?](https://mlsanity.com/paper/2002.08910/) (2020)
- [Siren's Song in the AI Ocean: A Survey on Hallucination in Large Language Models](https://mlsanity.com/paper/2309.01219/) (2023)
- [Active Retrieval Augmented Generation](https://mlsanity.com/paper/2305.06983/) (2023)
- [Entity-Based Knowledge Conflicts in Question Answering](https://mlsanity.com/paper/2109.05052/) (2021)
- [From Pretraining Data to Language Models to Downstream Tasks: Tracking the Trails of Political Biases Leading to Unfair NLP Models](https://mlsanity.com/paper/2305.08283/) (2023)
- [Can Language Models Solve Graph Problems in Natural Language?](https://mlsanity.com/paper/2305.10037/) (2023)
- [Memorizing Transformers](https://mlsanity.com/paper/2203.08913/) (2022)
- [Generated Knowledge Prompting for Commonsense Reasoning](https://mlsanity.com/paper/2110.08387/) (2021)
- [Factuality Enhanced Language Models for Open-Ended Text Generation](https://mlsanity.com/paper/2206.04624/) (2022)

## Built on by (most cited first, 30 of 50)

- [From System 1 to System 2: A Survey of Reasoning Large Language Models](https://mlsanity.com/paper/2502.17419/) (2025)
- [Know Your Limits: A Survey of Abstention in Large Language Models](https://mlsanity.com/paper/2407.18418/) (2024)
- [Modular Pluralism: Pluralistic Alignment via Multi-LLM Collaboration](https://mlsanity.com/paper/2406.15951/) (2024)
- [In-context Pretraining: Language Modeling Beyond Document Boundaries](https://mlsanity.com/paper/2310.10638/) (2023)
- [SealQA: Raising the Bar for Reasoning in Search-Augmented Language Models](https://mlsanity.com/paper/2506.01062/) (2025)
- [Large Language Models Hallucination: A Comprehensive Survey](https://mlsanity.com/paper/2510.06265/) (2025)
- [Rethinking Memory in LLM based Agents: Representations, Operations, and Emerging Topics](https://mlsanity.com/paper/2505.00675/) (2025)
- [Causal Prompting: Debiasing Large Language Model Prompting based on Front-Door Adjustment](https://mlsanity.com/paper/2403.02738/) (2024)
- [Taming Knowledge Conflicts in Language Models](https://mlsanity.com/paper/2503.10996/) (2025)
- [FactCHD: Benchmarking Fact-Conflicting Hallucination Detection](https://mlsanity.com/paper/2310.12086/) (2023)
- [Fundamental Problems With Model Editing: How Should Rational Belief Revision Work in LLMs?](https://mlsanity.com/paper/2406.19354/) (2024)
- [Benchmarking Multimodal Knowledge Conflict for Large Multimodal Models](https://mlsanity.com/paper/2505.19509/) (2025)
- [Correctness is not Faithfulness in RAG Attributions](https://mlsanity.com/paper/2412.18004/) (2024)
- [Analysing the Residual Stream of Language Models Under Knowledge Conflicts](https://mlsanity.com/paper/2410.16090/) (2024)
- [Understanding the Interplay between Parametric and Contextual Knowledge for Large Language Models](https://mlsanity.com/paper/2410.08414/) (2024)
- [Epistemic Alignment: A Mediating Framework for User-LLM Knowledge Delivery](https://mlsanity.com/paper/2504.01205/) (2025)
- [Beyond Math: Stories as a Testbed for Memorization-Constrained Reasoning in LLMs](https://mlsanity.com/paper/2412.14368/) (2024)
- [ParamMute: Suppressing Knowledge-Critical FFNs for Faithful Retrieval-Augmented Generation](https://mlsanity.com/paper/2502.15543/) (2025)
- [Evaluating the External and Parametric Knowledge Fusion of Large Language Models](https://mlsanity.com/paper/2405.19010/) (2024)
- [ChroKnowledge: Unveiling Chronological Knowledge of Language Models in Multiple Domains](https://mlsanity.com/paper/2410.09870/) (2024)
- [Mitigating Modal Imbalance in Multimodal Reasoning](https://mlsanity.com/paper/2510.02608/) (2025)
- [Helpful Agent Meets Deceptive Judge: Understanding Vulnerabilities in Agentic Workflows](https://mlsanity.com/paper/2506.03332/) (2025)
- [Dissecting Dissonance: Benchmarking Large Multimodal Models Against Self-Contradictory Instructions](https://mlsanity.com/paper/2408.01091/) (2024)
- [A Unified Definition of Hallucination: It's The World Model, Stupid!](https://mlsanity.com/paper/2512.21577/) (2025)
- [Investigating Tool-Memory Conflicts in Tool-Augmented LLMs](https://mlsanity.com/paper/2601.09760/) (2026)
- [ScienceMeter: Tracking Scientific Knowledge Updates in Language Models](https://mlsanity.com/paper/2505.24302/) (2025)
- [Aligning Knowledge Graphs and Language Models for Factual Accuracy](https://mlsanity.com/paper/2507.13411/) (2025)
- [From Faithfulness to Correctness: Generative Reward Models that Think Critically](https://mlsanity.com/paper/2509.25409/) (2025)
- [Contextual Drag: How Errors in the Context Affect LLM Reasoning](https://mlsanity.com/paper/2602.04288/) (2026)
- [EquiMem: Calibrating Shared Memory in Multi-Agent Debate via Game-Theoretic Equilibrium](https://mlsanity.com/paper/2605.09278/) (2026)

## Abstract (arXiv)

Large language models (LLMs) often encounter knowledge conflicts, scenarios where discrepancy arises between the internal parametric knowledge of LLMs and non-parametric information provided in the prompt context. In this work we ask what are the desiderata for LLMs when a knowledge conflict arises and whether existing LLMs fulfill them. We posit that LLMs should 1) identify knowledge conflicts, 2) pinpoint conflicting information segments, and 3) provide distinct answers or viewpoints in conflicting scenarios. To this end, we introduce an evaluation framework for simulating contextual knowledge conflicts and quantitatively evaluating to what extent LLMs achieve these goals. It includes diverse and complex situations of knowledge conflict, knowledge from diverse entities and domains, two synthetic conflict creation methods, and settings with progressively increasing difficulty to reflect realistic knowledge conflicts. Extensive experiments with the framework reveal that while LLMs perform well in identifying the existence of knowledge conflicts, they struggle to determine the specific conflicting knowledge and produce a response with distinct answers amidst conflicting information. To address these challenges, we propose new instruction-based approaches that augment LLMs to better achieve the three goals. Further analysis shows that abilities to tackle knowledge conflicts are greatly impacted by factors such as knowledge domain, while generating robust responses to knowledge conflict scenarios remains an open research question.