# ConflictBank: A Benchmark for Evaluating the Influence of Knowledge Conflicts in LLM

arXiv: https://arxiv.org/abs/2408.12076 · HTML: https://mlsanity.com/paper/2408.12076/

## Key facts

- Cited by 73 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 22 Aug 2024.

## Built on by (most cited first, 30 of 73)

- [Thinking with Images for Multimodal Reasoning: Foundations, Methods, and Future Frontiers](https://mlsanity.com/paper/2506.23918/) (2025)
- [OpenThinkIMG: Learning to Think with Images via Visual Tool Reinforcement Learning](https://mlsanity.com/paper/2505.08617/) (2025)
- [A Survey of Efficient Reasoning for Large Reasoning Models: Language, Multimodality, and Beyond](https://mlsanity.com/paper/2503.21614/) (2025)
- [PRMBench: A Fine-grained and Challenging Benchmark for Process-Level Reward Models](https://mlsanity.com/paper/2501.03124/) (2025)
- [SealQA: Raising the Bar for Reasoning in Search-Augmented Language Models](https://mlsanity.com/paper/2506.01062/) (2025)
- [Do as We Do, Not as You Think: the Conformity of Large Language Models](https://mlsanity.com/paper/2501.13381/) (2025)
- [Insight Over Sight: Exploring the Vision-Knowledge Conflicts in Multimodal LLMs](https://mlsanity.com/paper/2410.08145/) (2024)
- [Look, Compare, Decide: Alleviating Hallucination in Large Vision-Language Models via Multi-View Multi-Path Reasoning](https://mlsanity.com/paper/2408.17150/) (2024)
- [MMedPO: Aligning Medical Vision-Language Models with Clinical-Aware Multimodal Preference Optimization](https://mlsanity.com/paper/2412.06141/) (2024)
- [Benchmarking Multimodal Knowledge Conflict for Large Multimodal Models](https://mlsanity.com/paper/2505.19509/) (2025)
- [Analysing the Residual Stream of Language Models Under Knowledge Conflicts](https://mlsanity.com/paper/2410.16090/) (2024)
- [NarraBench: A Comprehensive Framework for Narrative Benchmarking](https://mlsanity.com/paper/2510.09869/) (2025)
- [Beyond Math: Stories as a Testbed for Memorization-Constrained Reasoning in LLMs](https://mlsanity.com/paper/2412.14368/) (2024)
- [Whose Facts Win? LLM Source Preferences under Knowledge Conflicts](https://mlsanity.com/paper/2601.03746/) (2026)
- [SURf: Teaching Large Vision-Language Models to Selectively Utilize Retrieved Information](https://mlsanity.com/paper/2409.14083/) (2024)
- [Do Automatic Factuality Metrics Measure Factuality? A Critical Evaluation](https://mlsanity.com/paper/2411.16638/) (2024)
- [Resisting Contextual Interference in RAG via Parametric-Knowledge Reinforcement](https://mlsanity.com/paper/2506.05154/) (2025)
- [From Head to Tail: Towards Balanced Representation in Large Vision-Language Models through Adaptive Data Calibration](https://mlsanity.com/paper/2503.12821/) (2025)
- [A Multimodal Multi-Agent Framework for Radiology Report Generation](https://mlsanity.com/paper/2505.09787/) (2025)
- [Can Large Language Models Match the Conclusions of Systematic Reviews?](https://mlsanity.com/paper/2505.22787/) (2025)
- [Mitigating Modal Imbalance in Multimodal Reasoning](https://mlsanity.com/paper/2510.02608/) (2025)
- [Discovering Knowledge Deficiencies of Language Models on Massive Knowledge Base](https://mlsanity.com/paper/2503.23361/) (2025)
- [A Comprehensive Review of AI Agents: Transforming Possibilities in Technology and Beyond](https://mlsanity.com/paper/2508.11957/) (2025)
- [Seeing through the Conflict: Transparent Knowledge Conflict Handling in Retrieval-Augmented Generation](https://mlsanity.com/paper/2601.06842/) (2026)
- [Conditional [MASK] Discrete Diffusion Language Model](https://mlsanity.com/paper/2411.06438/) (2024)
- [How does Misinformation Affect Large Language Model Behaviors and Preferences?](https://mlsanity.com/paper/2505.21608/) (2025)
- [Consensus or Conflict? Fine-Grained Evaluation of Conflicting Answers in Question-Answering](https://mlsanity.com/paper/2508.12355/) (2025)
- [After Retrieval, Before Generation: Enhancing the Trustworthiness of Large Language Models in Retrieval-Augmented Generation](https://mlsanity.com/paper/2505.17118/) (2025)
- [Format as a Prior: Quantifying and Analyzing Bias in LLMs for Heterogeneous Data](https://mlsanity.com/paper/2508.15793/) (2025)
- [Flying Pigs, FaR and Beyond: Evaluating LLM Reasoning in Counterfactual Worlds](https://mlsanity.com/paper/2505.22318/) (2025)

## Abstract (arXiv)

Large language models (LLMs) have achieved impressive advancements across numerous disciplines, yet the critical issue of knowledge conflicts, a major source of hallucinations, has rarely been studied. Only a few research explored the conflicts between the inherent knowledge of LLMs and the retrieved contextual knowledge. However, a thorough assessment of knowledge conflict in LLMs is still missing. Motivated by this research gap, we present ConflictBank, the first comprehensive benchmark developed to systematically evaluate knowledge conflicts from three aspects: (i) conflicts encountered in retrieved knowledge, (ii) conflicts within the models' encoded knowledge, and (iii) the interplay between these conflict forms. Our investigation delves into four model families and twelve LLM instances, meticulously analyzing conflicts stemming from misinformation, temporal discrepancies, and semantic divergences. Based on our proposed novel construction framework, we create 7,453,853 claim-evidence pairs and 553,117 QA pairs. We present numerous findings on model scale, conflict causes, and conflict types. We hope our ConflictBank benchmark will help the community better understand model behavior in conflicts and develop more reliable LLMs.