# ECon: On the Detection and Resolution of Evidence Conflicts

arXiv: https://arxiv.org/abs/2410.04068 · HTML: https://mlsanity.com/paper/2410.04068/

## Key facts

- Cited by 18 papers in the map (data through 14 Oct 2026).
- Builds on 34 of its 55 references that are in the map.
- First posted on arXiv 5 Oct 2024.

## Builds on (most cited first, 30 of 34)

- [Dense Passage Retrieval for Open-Domain Question Answering](https://mlsanity.com/paper/2004.04906/) (2020)
- [Sparks of Artificial General Intelligence: Early experiments with GPT-4](https://mlsanity.com/paper/2303.12712/) (2023)
- [DeBERTa: Decoding-enhanced BERT with Disentangled Attention](https://mlsanity.com/paper/2006.03654/) (2020)
- [FActScore: Fine-grained Atomic Evaluation of Factual Precision in Long Form Text Generation](https://mlsanity.com/paper/2305.14251/) (2023)
- [Large Language Models Can Be Easily Distracted by Irrelevant Context](https://mlsanity.com/paper/2302.00093/) (2023)
- [Reading Wikipedia to Answer Open-Domain Questions](https://mlsanity.com/paper/1704.00051/) (2017)
- [Did Aristotle Use a Laptop? A Question Answering Benchmark with Implicit Reasoning Strategies](https://mlsanity.com/paper/2101.02235/) (2021)
- [Latent Retrieval for Weakly Supervised Open Domain Question Answering](https://mlsanity.com/paper/1906.00300/) (2019)
- [Adversarial Examples for Evaluating Reading Comprehension Systems](https://mlsanity.com/paper/1707.07328/) (2017)
- [The Web as a Knowledge-base for Answering Complex Questions](https://mlsanity.com/paper/1803.06643/) (2018)
- [AmbigQA: Answering Ambiguous Open-domain Questions](https://mlsanity.com/paper/2004.10645/) (2020)
- [Entity-Based Knowledge Conflicts in Question Answering](https://mlsanity.com/paper/2109.05052/) (2021)
- [Adaptive Chameleon or Stubborn Sloth: Revealing the Behavior of Large Language Models in Knowledge Conflicts](https://mlsanity.com/paper/2305.13300/) (2023)
- [AlignScore: Evaluating Factual Consistency with a Unified Alignment Function](https://mlsanity.com/paper/2305.16739/) (2023)
- [Analyzing Leakage of Personally Identifiable Information in Language Models](https://mlsanity.com/paper/2302.00539/) (2023)
- [Mathematical Capabilities of ChatGPT](https://mlsanity.com/paper/2301.13867/) (2023)
- [MiniCheck: Efficient Fact-Checking of LLMs on Grounding Documents](https://mlsanity.com/paper/2404.10774/) (2024)
- [Can LLM-Generated Misinformation Be Detected?](https://mlsanity.com/paper/2309.13788/) (2023)
- [Generative Language Models and Automated Influence Operations: Emerging Threats and Potential Mitigations](https://mlsanity.com/paper/2301.04246/) (2023)
- [SituatedQA: Incorporating Extra-Linguistic Contexts into QA](https://mlsanity.com/paper/2109.06157/) (2021)
- [Rich Knowledge Sources Bring Complex Knowledge Conflicts: Recalibrating Models to Reflect Conflicting Evidence](https://mlsanity.com/paper/2210.13701/) (2022)
- [Complex Claim Verification with Evidence Retrieved in the Wild](https://mlsanity.com/paper/2305.11859/) (2023)
- [ChatGPT: The End of Online Exam Integrity?](https://mlsanity.com/paper/2212.09292/) (2022)
- [NegotiationToM: A Benchmark for Stress-testing Machine Theory of Mind on Negotiation Surrounding](https://mlsanity.com/paper/2404.13627/) (2024)
- [PrivLM-Bench: A Multi-level Privacy Evaluation Benchmark for Language Models](https://mlsanity.com/paper/2311.04044/) (2023)
- [Text-Tuple-Table: Towards Information Integration in Text-to-Table Generation via Global Tuple Extraction](https://mlsanity.com/paper/2404.14215/) (2024)
- [CANDLE: Iterative Conceptualization and Instantiation Distillation from Large Language Models for Commonsense Reasoning](https://mlsanity.com/paper/2401.07286/) (2024)
- [StoryAnalogy: Deriving Story-level Analogies from Large Language Models to Unlock Analogical Understanding](https://mlsanity.com/paper/2310.12874/) (2023)
- [OpenResearcher: Unleashing AI for Accelerated Scientific Research](https://mlsanity.com/paper/2408.06941/) (2024)
- [DiscoPrompt: Path Prediction Prompt Tuning for Implicit Discourse Relation Recognition](https://mlsanity.com/paper/2305.03973/) (2023)

## Built on by (most cited first, 18 of 18)

- [SealQA: Raising the Bar for Reasoning in Search-Augmented Language Models](https://mlsanity.com/paper/2506.01062/) (2025)
- [DRAGged into Conflicts: Detecting and Addressing Conflicting Sources in Search-Augmented LLMs](https://mlsanity.com/paper/2506.08500/) (2025)
- [Large Language Models in Argument Mining: A Survey](https://mlsanity.com/paper/2506.16383/) (2025)
- [ConInstruct: Evaluating Large Language Models on Conflict Detection and Resolution in Instructions](https://mlsanity.com/paper/2511.14342/) (2025)
- [LegalWiz: A Multi-Agent Generation Framework for Contradiction Detection in Legal Documents](https://mlsanity.com/paper/2510.03418/) (2025)
- [MAGIC: A Multi-Hop and Graph-Based Benchmark for Inter-Context Conflicts in Retrieval-Augmented Generation](https://mlsanity.com/paper/2507.21544/) (2025)
- [Contradiction Detection in RAG Systems: Evaluating LLMs as Context Validators for Improved Information Consistency](https://mlsanity.com/paper/2504.00180/) (2025)
- [Conflicts in Texts: Data, Implications and Challenges](https://mlsanity.com/paper/2504.19472/) (2025)
- [XToM: Exploring the Multilingual Theory of Mind for Large Language Models](https://mlsanity.com/paper/2506.02461/) (2025)
- [Structuring the Unstructured: A Systematic Review of Text-to-Structure Generation for Agentic AI with a Universal Evaluation Framework](https://mlsanity.com/paper/2508.12257/) (2025)
- [Coalition Formation in LLM Agent Networks: Stability Analysis and Convergence Guarantees](https://mlsanity.com/paper/2604.14386/) (2026)
- [InteGround: On the Evaluation of Verification and Retrieval Planning in Integrative Grounding](https://mlsanity.com/paper/2509.16534/) (2025)
- [On Finding Inconsistencies in Documents](https://mlsanity.com/paper/2512.18601/) (2025)
- [Cherry-pick Override: LLM Judges Under-use the Non-Directional Verdicts Their Contract Authorizes](https://mlsanity.com/paper/2606.07834/) (2026)
- [ConflictScore: Identifying and Measuring How Language Models Handle Conflicting Evidence](https://mlsanity.com/paper/2606.26437/) (2026)
- [When Text and Numbers Disagree: Evidence Arbitration in Large Language Models](https://mlsanity.com/paper/2608.20116/) (2026)
- [CoVer: Conflict-Aware Claim Verification](https://mlsanity.com/paper/2609.00508/) (2026)
- [Overlap, Unique and Conflict: Can LLMs Extract What They Can Recognize?](https://mlsanity.com/paper/2609.38799/) (2026)

## Abstract (arXiv)

The rise of large language models (LLMs) has significantly influenced the quality of information in decision-making systems, leading to the prevalence of AI-generated content and challenges in detecting misinformation and managing conflicting information, or "inter-evidence conflicts." This study introduces a method for generating diverse, validated evidence conflicts to simulate real-world misinformation scenarios. We evaluate conflict detection methods, including Natural Language Inference (NLI) models, factual consistency (FC) models, and LLMs, on these conflicts (RQ1) and analyze LLMs' conflict resolution behaviors (RQ2). Our key findings include: (1) NLI and LLM models exhibit high precision in detecting answer conflicts, though weaker models suffer from low recall; (2) FC models struggle with lexically similar answer conflicts, while NLI and LLM models handle these better; and (3) stronger models like GPT-4 show robust performance, especially with nuanced conflicts. For conflict resolution, LLMs often favor one piece of conflicting evidence without justification and rely on internal knowledge if they have prior beliefs.