# AmbiFC: Fact-Checking Ambiguous Claims with Evidence

arXiv: https://arxiv.org/abs/2104.00640 · HTML: https://mlsanity.com/paper/2104.00640/

## Key facts

- Cited by 38 papers in the map (data through 14 Oct 2026).
- Builds on 21 of its 68 references that are in the map.
- First posted on arXiv 1 Apr 2021.

## Builds on (most cited first, 21 of 21)

- [Distilling the Knowledge in a Neural Network](https://mlsanity.com/paper/1503.02531/) (2015)
- [On Calibration of Modern Neural Networks](https://mlsanity.com/paper/1706.04599/) (2017)
- [A Broad-Coverage Challenge Corpus for Sentence Understanding through Inference](https://mlsanity.com/paper/1704.05426/) (2017)
- [BoolQ: Exploring the Surprising Difficulty of Natural Yes/No Questions](https://mlsanity.com/paper/1905.10044/) (2019)
- [FEVER: a large-scale dataset for Fact Extraction and VERification](https://mlsanity.com/paper/1803.05355/) (2018)
- [Fact or Fiction: Verifying Scientific Claims](https://mlsanity.com/paper/2004.14974/) (2020)
- [TabFact: A Large-scale Dataset for Table-based Fact Verification](https://mlsanity.com/paper/1909.02164/) (2019)
- [AmbigQA: Answering Ambiguous Open-domain Questions](https://mlsanity.com/paper/2004.10645/) (2020)
- [Calibration of Pre-trained Transformers](https://mlsanity.com/paper/2003.07892/) (2020)
- [Human uncertainty makes classification more robust](https://mlsanity.com/paper/1908.07086/) (2019)
- [Get Your Vitamin C! Robust Fact Verification with Contrastive Evidence](https://mlsanity.com/paper/2103.08541/) (2021)
- [HoVer: A Dataset for Many-Hop Fact Extraction And Claim Verification](https://mlsanity.com/paper/2011.03088/) (2020)
- [CLIMATE-FEVER: A Dataset for Verification of Real-World Climate Claims](https://mlsanity.com/paper/2012.00614/) (2020)
- [MultiFC: A Real-World Multi-Domain Dataset for Evidence-Based Fact Checking of Claims](https://mlsanity.com/paper/1909.03242/) (2019)
- [What Can We Learn from Collective Human Opinions on Natural Language Inference Data?](https://mlsanity.com/paper/2010.03532/) (2020)
- [Towards Debiasing Fact Verification Models](https://mlsanity.com/paper/1908.05267/) (2019)
- [Automated Fact Checking: Task formulations, methods and future directions](https://mlsanity.com/paper/1806.07687/) (2018)
- [Seeing Things from a Different Angle: Discovering Diverse Perspectives about Claims](https://mlsanity.com/paper/1906.03538/) (2019)
- [A Richly Annotated Corpus for Different Tasks in Automated Fact-Checking](https://mlsanity.com/paper/1911.01214/) (2019)
- [Argumentation Mining in User-Generated Web Discourse](https://mlsanity.com/paper/1601.02403/) (2016)
- [Why Does a Visual Question Have Different Answers?](https://mlsanity.com/paper/1908.04342/) (2019)

## Built on by (most cited first, 30 of 38)

- [MiniCheck: Efficient Fact-Checking of LLMs on Grounding Documents](https://mlsanity.com/paper/2404.10774/) (2024)
- [SCITAB: A Challenging Benchmark for Compositional Reasoning and Claim Verification on Scientific Tables](https://mlsanity.com/paper/2305.13186/) (2023)
- [DEFAME: Dynamic Evidence-based FAct-checking with Multimodal Experts](https://mlsanity.com/paper/2412.10510/) (2024)
- [A Closer Look at Claim Decomposition](https://mlsanity.com/paper/2403.11903/) (2024)
- [Can Large Language Models Capture Dissenting Human Voices?](https://mlsanity.com/paper/2305.13788/) (2023)
- [What Can Natural Language Processing Do for Peer Review?](https://mlsanity.com/paper/2405.06563/) (2024)
- [Generative Large Language Models in Automated Fact-Checking: A Survey](https://mlsanity.com/paper/2407.02351/) (2024)
- [FinDVer: Explainable Claim Verification over Long and Hybrid-Content Financial Documents](https://mlsanity.com/paper/2411.05764/) (2024)
- [Towards Effective Extraction and Evaluation of Factual Claims](https://mlsanity.com/paper/2502.10855/) (2025)
- [SciVer: Evaluating Foundation Models for Multimodal Scientific Claim Verification](https://mlsanity.com/paper/2506.15569/) (2025)
- [Interpreting Predictive Probabilities: Model Confidence or Human Label Variation?](https://mlsanity.com/paper/2402.16102/) (2024)
- [Verifying the Verifiers: Unveiling Pitfalls and Potentials in Fact Verifiers](https://mlsanity.com/paper/2506.13342/) (2025)
- [Enhancing Systematic Decompositional Natural Language Inference Using Informal Logic](https://mlsanity.com/paper/2402.14798/) (2024)
- [DeepFact: Co-Evolving Benchmarks and Agents for Deep Research Factuality](https://mlsanity.com/paper/2603.05912/) (2026)
- [CHECKWHY: Causal Fact Verification via Argument Structure](https://mlsanity.com/paper/2408.10918/) (2024)
- [A Reality Check on Context Utilisation for Retrieval-Augmented Generation](https://mlsanity.com/paper/2412.17031/) (2024)
- [Beyond Translation: LLM-Based Data Generation for Multilingual Fact-Checking](https://mlsanity.com/paper/2502.15419/) (2025)
- [Optimal-Agent-Selection: State-Aware Routing Framework for Efficient Multi-Agent Collaboration](https://mlsanity.com/paper/2511.02200/) (2025)
- [VeriTaS: The First Dynamic Benchmark for Multimodal Automated Fact-Checking](https://mlsanity.com/paper/2601.08611/) (2026)
- [Facts Fade Fast: Evaluating Memorization of Outdated Medical Knowledge in Large Language Models](https://mlsanity.com/paper/2509.04304/) (2025)
- [Sightation Counts: Leveraging Sighted User Feedback in Building a BLV-aligned Dataset of Diagram Descriptions](https://mlsanity.com/paper/2503.13369/) (2025)
- [Beyond Frameworks: Unpacking Collaboration Strategies in Multi-Agent Systems](https://mlsanity.com/paper/2505.12467/) (2025)
- [VeriTrail: Closed-Domain Hallucination Detection with Traceability](https://mlsanity.com/paper/2505.21786/) (2025)
- [Decide less, communicate more: On the construct validity of end-to-end fact-checking in medicine](https://mlsanity.com/paper/2506.20876/) (2025)
- [DecomposeRL: Learning to Ask Useful, Informative, and Diverse Questions for Semi-Supervised, Traceable Claim Verification](https://mlsanity.com/paper/2605.27858/) (2026)
- [CRAVE: A Conflicting Reasoning Approach for Explainable Claim Verification Using LLMs](https://mlsanity.com/paper/2504.14905/) (2025)
- [Capturing Symmetry and Antisymmetry in Language Models through Symmetry-Aware Training Objectives](https://mlsanity.com/paper/2504.16312/) (2025)
- [The Psychology of Falsehood: A Human-Centric Survey of Misinformation Detection](https://mlsanity.com/paper/2509.15896/) (2025)
- [The Gray Zone of Faithfulness: Taming Ambiguity in Unfaithfulness Detection](https://mlsanity.com/paper/2510.21118/) (2025)
- [Can LLMs extract human-like fine-grained evidence for evidence-based fact-checking?](https://mlsanity.com/paper/2511.21401/) (2025)

## Abstract (arXiv)

Automated fact-checking systems verify claims against evidence to predict their veracity. In real-world scenarios, the retrieved evidence may not unambiguously support or refute the claim and yield conflicting but valid interpretations. Existing fact-checking datasets assume that the models developed with them predict a single veracity label for each claim, thus discouraging the handling of such ambiguity. To address this issue we present AmbiFC, a fact-checking dataset with 10k claims derived from real-world information needs. It contains fine-grained evidence annotations of 50k passages from 5k Wikipedia pages. We analyze the disagreements arising from ambiguity when comparing claims against evidence in AmbiFC, observing a strong correlation of annotator disagreement with linguistic phenomena such as underspecification and probabilistic reasoning. We develop models for predicting veracity handling this ambiguity via soft labels and find that a pipeline that learns the label distribution for sentence-level evidence selection and veracity prediction yields the best performance. We compare models trained on different subsets of AmbiFC and show that models trained on the ambiguous instances perform better when faced with the identified linguistic phenomena.