mlsanityData to 14 Oct 2026

ConflictBank: A Benchmark for Evaluating the Influence of Knowledge Conflicts in LLM

arXiv 2408.12076 ↗First posted 22 Aug 2024cs.CL cs.AIcited by 73

Not on the map. Why?

Abstract (arXiv)

Large language models (LLMs) have achieved impressive advancements across numerous disciplines, yet the critical issue of knowledge conflicts, a major source of hallucinations, has rarely been studied. Only a few research explored the conflicts between the inherent knowledge of LLMs and the retrieved contextual knowledge. However, a thorough assessment of knowledge conflict in LLMs is still missing. Motivated by this research gap, we present ConflictBank, the first comprehensive benchmark developed to systematically evaluate knowledge conflicts from three aspects: (i) conflicts encountered in retrieved knowledge, (ii) conflicts within the models' encoded knowledge, and (iii) the interplay between these conflict forms. Our investigation delves into four model families and twelve LLM instances, meticulously analyzing conflicts stemming from misinformation, temporal discrepancies, and semantic divergences. Based on our proposed novel construction framework, we create 7,453,853 claim-evidence pairs and 553,117 QA pairs. We present numerous findings on model scale, conflict causes, and conflict types. We hope our ConflictBank benchmark will help the community better understand model behavior in conflicts and develop more reliable LLMs.

Key facts. Cited by 73 papers in the map (data through 14 Oct 2026). First posted on arXiv 22 Aug 2024.

Builds on

0 of 0 references.

No references here yet.

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 8; 2025: 33; 2026: 32.

73 papers.

Sort:

By area:

  1. 2025Thinking with Images for Multimodal Reasoning: Foundations, Methods, and Future Frontierscited by 199
  2. 2025OpenThinkIMG: Learning to Think with Images via Visual Tool Reinforcement Learningcited by 163
  3. 2025A Survey of Efficient Reasoning for Large Reasoning Models: Language, Multimodality, and Beyondcited by 132
  4. 2025PRMBench: A Fine-grained and Challenging Benchmark for Process-Level Reward Modelscited by 99
  5. 2025SealQA: Raising the Bar for Reasoning in Search-Augmented Language Modelscited by 75
  6. 2025Do as We Do, Not as You Think: the Conformity of Large Language Modelscited by 53
  7. 2024Insight Over Sight: Exploring the Vision-Knowledge Conflicts in Multimodal LLMscited by 38
  8. 2024Look, Compare, Decide: Alleviating Hallucination in Large Vision-Language Models via Multi-View Multi-Path Reasoningcited by 23
  9. 2024MMedPO: Aligning Medical Vision-Language Models with Clinical-Aware Multimodal Preference Optimizationcited by 21
  10. 2025Benchmarking Multimodal Knowledge Conflict for Large Multimodal Modelscited by 19
  11. 2024Analysing the Residual Stream of Language Models Under Knowledge Conflictscited by 17
  12. 2025NarraBench: A Comprehensive Framework for Narrative Benchmarkingcited by 13
38 more
  1. 2024Beyond Math: Stories as a Testbed for Memorization-Constrained Reasoning in LLMscited by 12
  2. 2026Whose Facts Win? LLM Source Preferences under Knowledge Conflictscited by 12
  3. 2024SURf: Teaching Large Vision-Language Models to Selectively Utilize Retrieved Informationcited by 11
  4. 2024Do Automatic Factuality Metrics Measure Factuality? A Critical Evaluationcited by 11
  5. 2025Resisting Contextual Interference in RAG via Parametric-Knowledge Reinforcementcited by 11
  6. 2025From Head to Tail: Towards Balanced Representation in Large Vision-Language Models through Adaptive Data Calibrationcited by 8
  7. 2025A Multimodal Multi-Agent Framework for Radiology Report Generationcited by 8
  8. 2025Can Large Language Models Match the Conclusions of Systematic Reviews?cited by 8
  9. 2025Mitigating Modal Imbalance in Multimodal Reasoningcited by 8
  10. 2025Discovering Knowledge Deficiencies of Language Models on Massive Knowledge Basecited by 6
  11. 2025A Comprehensive Review of AI Agents: Transforming Possibilities in Technology and Beyondcited by 6
  12. 2026Seeing through the Conflict: Transparent Knowledge Conflict Handling in Retrieval-Augmented Generationcited by 6
  13. 2024Conditional [MASK] Discrete Diffusion Language Modelcited by 5
  14. 2025How does Misinformation Affect Large Language Model Behaviors and Preferences?cited by 5
  15. 2025Consensus or Conflict? Fine-Grained Evaluation of Conflicting Answers in Question-Answeringcited by 5
  16. 2025After Retrieval, Before Generation: Enhancing the Trustworthiness of Large Language Models in Retrieval-Augmented Generationcited by 4
  17. 2025Format as a Prior: Quantifying and Analyzing Bias in LLMs for Heterogeneous Datacited by 4
  18. 2025Flying Pigs, FaR and Beyond: Evaluating LLM Reasoning in Counterfactual Worldscited by 3
  19. 2026Faithfulness vs. Safety: Evaluating LLM Behavior Under Counterfactual Medical Evidencecited by 3
  20. 2025Fact or Guesswork? Evaluating Large Language Models' Medical Knowledge with Structured One-Hop Judgmentscited by 2
  21. 2025FactReasoner: A Probabilistic Approach to Long-Form Factuality Assessment for Large Language Modelscited by 2
  22. 2025Micro-Act: Mitigating Knowledge Conflict in LLM-based RAG via Actionable Self-Reasoningcited by 2
  23. 2025Comprehensiveness Metrics for Automatic Evaluation of Factual Recall in Text Generationcited by 2
  24. 2025When Facts Change: Probing LLMs on Evolving Knowledge with evolveQAcited by 2
  25. 2026Temporal Fact Conflicts in LLMs: Reproducibility Insights from Unifying DYNAMICQA and MULANcited by 2
  26. 2026The Override Gap: A Magnitude Account of Knowledge Conflict Failure in Hypernetwork-Based Instant LLM Adaptationcited by 2
  27. 2026Architecture Matters: Comparing RAG Systems under Knowledge Base Poisoningcited by 2
  28. 2026Mitigating Context-Memory Conflicts in LLMs through Dynamic Cognitive Reconciliation Decodingcited by 2
  29. 2026HalluWorld: A Controlled Benchmark for Hallucination via Reference World Modelscited by 2
  30. 2025Resolving Knowledge Conflicts in Domain-specific Data Selection: A Case Study on Medical Instruction-tuningcited by 1
  31. 2025A Multi-Agent System for Complex Reasoning in Radiology Visual Question Answeringcited by 1
  32. 2025MillStone: How Open-Minded Are LLMs?cited by 1
  33. 2025A Multifaceted Analysis of Negative Bias in Large Language Models through the Lens of Parametric Knowledgecited by 1
  34. 2026REAL: Resolving Knowledge Conflicts in Knowledge-Intensive Visual Question Answering via Reasoning-Pivot Alignmentcited by 1
  35. 2026Diagnosing Knowledge Conflict in Multimodal Long-Chain Reasoningcited by 1
  36. 2026Diagnosing LLM Arbitration Behavior over Pre-evidence Epistemic States in RAG-based Fact-Checkingcited by 1
  37. 2026Navigating Unreliable Parametric and Contextual Knowledge: Explicit Knowledge Conflict Resolution for LLM Inferencecited by 1
  38. 2026ReasoningLens: Hierarchical Visualization and Diagnostic Auditing for Large Reasoning Modelscited by 1

50 most cited of 73 All 73 (JSON)