Overlap, Unique and Conflict: Can LLMs Extract What They Can Recognize?
Not on the map. Why?
Abstract (arXiv)
Understanding multi-perspective alternative narratives requires identifying how their information agrees, conflicts, or differs across sources. Existing work on cross-text relations largely focuses on categorizing relations between predefined text pairs, such as entailment or contradiction, rather than directly extracting such information from full narratives. To address this gap, we introduce Overlap-Unique-Conflict (OUC) extraction, a cross-narrative task that extracts all overlapping, conflicting, and unique clauses from two narratives. To support this study, we construct a benchmark of approximately 22K narrative pairs and 140K OUC instances spanning factual, argumentative, and political discourse. Evaluating 14 open-source LLMs (0.6B-35B), we find that unique information is far easier to extract than overlap and conflict: the strongest model, Gemma-4-31B, reaches only 61.13% F1-score on overlap and 48.58% on conflict, against more than 75% on unique. Further diagnostic analysis reveals that this difficulty does not stem from relation recognition alone, but rather from a failure to pair and extract the corresponding clauses from full narratives, especially in smaller models. Nevertheless, learning these extractions with task-specific supervision narrows the gap considerably: a fine-tuned Qwen-3-8B gains 15-28% absolute over its baseline and surpasses models roughly four times its size (e.g., Qwen-3.6-35B) on several tasks. Even so, overlap and conflict remain well below satisfactory, leaving cross-narrative clause extraction an open challenge.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 39 of its 64 references that are in the map. First posted on arXiv 30 Sep 2026.
Builds on
39 of 64 references.
References · 39
- 2024The Llama 3 Herd of Modelscited by 14,715
- 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 9,379
- 2025Qwen3 Technical Reportcited by 8,820
- 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 8,065
- 2023Efficient Memory Management for Large Language Model Serving with PagedAttentioncited by 5,944
- 2017A Broad-Coverage Challenge Corpus for Sentence Understanding through Inferencecited by 2,682
- 2024HybridFlow: A Flexible and Efficient RLHF Frameworkcited by 2,057
- 2015A large annotated corpus for learning natural language inferencecited by 1,847
- 2018FEVER: a large-scale dataset for Fact Extraction and VERificationcited by 1,375
- 2019Adversarial NLI: A New Benchmark for Natural Language Understandingcited by 807
- 2024Phi-4 Technical Reportcited by 711
- 2024LongBench v2: Towards Deeper Understanding and Reasoning on Realistic Long-context Multitaskscited by 379
27 more
- 2026Gemma 4 Technical Reportcited by 259
- 2024On LLMs-Driven Synthetic Data Generation, Curation, and Evaluation: A Surveycited by 244
- 2024Leave No Document Behind: Benchmarking Long-Context LLMs with Extended Multi-Doc QAcited by 122
- 2025Reinforcement Learning with Verifiable Rewards: GRPO's Effective Loss, Dynamics, and Success Amplificationcited by 110
- 2024What Evidence Do Language Models Find Convincing?cited by 75
- 2020A Large-Scale Multi-Document Summarization Dataset from the Wikipedia Current Events Portalcited by 72
- 2025NVIDIA Nemotron 3: Efficient and Open Intelligencecited by 65
- 2025Synthetic Data Generation Using Large Language Models: Advances in Text and Codecited by 61
- 2024DataDreamer: A Tool for Synthetic Data Generation and Reproducible LLM Workflowscited by 43
- 2024If in a Crowdsourced Data Annotation Pipeline, a GPT-4cited by 38
- 2025Trust, But Verify: A Self-Verification Approach to Reinforcement Learning with Verifiable Rewardscited by 38
- 2024FanOutQA: A Multi-Hop, Multi-Document Question Answering Benchmark for Large Language Modelscited by 29
- 2023Summarizing Multiple Documents with Conversational Structure for Meta-Review Generationcited by 27
- 2018Neural Network Models for Paraphrase Identification, Semantic Textual Similarity, Natural Language Inference, and Question Answeringcited by 23
- 2024DataGen: Unified Synthetic Dataset Generation via Large Language Modelscited by 23
- 2024ECon: On the Detection and Resolution of Evidence Conflictscited by 18
- 2021QA-Align: Representing Cross-Text Content Overlap by Aligning Question-Answer Propositionscited by 16
- 2021MultiOpEd: A Corpus of Multi-Perspective News Editorialscited by 13
- 2025Lessons from Training Grounded LLMs with Verifiable Rewardscited by 8
- 2026ExecVerify: White-Box RL with Verifiable Stepwise Rewards for Code Execution Reasoningcited by 6
- 2026Skill-Aware Data Selection and Fine-Tuning for Data-Efficient Reasoning Distillationcited by 5
- 2024Synergetic Event Understanding: A Collaborative Approach to Cross-Document Event Coreference Resolution with Large Language Modelscited by 3
- 2024Towards Better Graph-based Cross-document Relation Extraction via Non-bridge Entity Enhancement and Prediction Debiasingcited by 3
- 2025DRIVE: Data Curation Best Practices for Reinforcement Learning with Verifiable Reward in Competitive Code Generationcited by 3
- 2024Knowledge-Driven Cross-Document Relation Extractioncited by 2
- 2024Harvesting Events from Multiple Sources: Towards a Cross-Document Event Extraction Paradigmcited by 2
- 2024AIDE: Attribute-Guided MultI-Hop Data Expansion for Data Scarcity in Task-Specific Fine-tuningcited by 2
Built on by
Not cited yet.