# FaVIQ: FAct Verification from Information-seeking Questions

arXiv: https://arxiv.org/abs/2107.02153 · HTML: https://mlsanity.com/paper/2107.02153/

## Key facts

- Cited by 36 papers in the map (data through 14 Oct 2026).
- Builds on 34 of its 45 references that are in the map.
- First posted on arXiv 5 Jul 2021.

## Builds on (most cited first, 30 of 34)

- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://mlsanity.com/paper/1910.10683/) (2019)
- [BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension](https://mlsanity.com/paper/1910.13461/) (2019)
- [On Calibration of Modern Neural Networks](https://mlsanity.com/paper/1706.04599/) (2017)
- [Dense Passage Retrieval for Open-Domain Question Answering](https://mlsanity.com/paper/2004.04906/) (2020)
- [HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answering](https://mlsanity.com/paper/1809.09600/) (2018)
- [SuperGLUE: A Stickier Benchmark for General-Purpose Language Understanding Systems](https://mlsanity.com/paper/1905.00537/) (2019)
- [Calibrate Before Use: Improving Few-Shot Performance of Language Models](https://mlsanity.com/paper/2102.09690/) (2021)
- [FEVER: a large-scale dataset for Fact Extraction and VERification](https://mlsanity.com/paper/1803.05355/) (2018)
- [Reading Wikipedia to Answer Open-Domain Questions](https://mlsanity.com/paper/1704.00051/) (2017)
- [Latent Retrieval for Weakly Supervised Open Domain Question Answering](https://mlsanity.com/paper/1906.00300/) (2019)
- [KILT: a Benchmark for Knowledge Intensive Language Tasks](https://mlsanity.com/paper/2009.02252/) (2020)
- [Fact or Fiction: Verifying Scientific Claims](https://mlsanity.com/paper/2004.14974/) (2020)
- [AmbigQA: Answering Ambiguous Open-domain Questions](https://mlsanity.com/paper/2004.10645/) (2020)
- [Calibration of Pre-trained Transformers](https://mlsanity.com/paper/2003.07892/) (2020)
- [Selective Question Answering under Domain Shift](https://mlsanity.com/paper/2006.09462/) (2020)
- [Get Your Vitamin C! Robust Fact Verification with Contrastive Evidence](https://mlsanity.com/paper/2103.08541/) (2021)
- [HoVer: A Dataset for Many-Hop Fact Extraction And Claim Verification](https://mlsanity.com/paper/2011.03088/) (2020)
- [MultiFC: A Real-World Multi-Domain Dataset for Evidence-Based Fact Checking of Claims](https://mlsanity.com/paper/1909.03242/) (2019)
- [Towards Debiasing Fact Verification Models](https://mlsanity.com/paper/1908.05267/) (2019)
- [Multi-hop Reading Comprehension through Question Decomposition and Rescoring](https://mlsanity.com/paper/1906.02916/) (2019)
- [The Fact Extraction and VERification (FEVER) Shared Task](https://mlsanity.com/paper/1811.10971/) (2018)
- [Transforming Question Answering Datasets Into Natural Language Inference Datasets](https://mlsanity.com/paper/1809.02922/) (2018)
- [A Richly Annotated Corpus for Different Tasks in Automated Fact-Checking](https://mlsanity.com/paper/1911.01214/) (2019)
- [Fool Me Twice: Entailment from Wikipedia Gamification](https://mlsanity.com/paper/2104.04725/) (2021)
- [Can NLI Models Verify QA Systems' Predictions?](https://mlsanity.com/paper/2104.08731/) (2021)
- [Evidence-based Factual Error Correction](https://mlsanity.com/paper/2012.15788/) (2020)
- [Multi-task Retrieval for Knowledge-Intensive Tasks](https://mlsanity.com/paper/2101.00117/) (2021)
- [Zero-shot Fact Verification by Claim Generation](https://mlsanity.com/paper/2105.14682/) (2021)
- [Which Linguist Invented the Lightbulb? Presupposition Verification for Question-Answering](https://mlsanity.com/paper/2101.00391/) (2021)

## Built on by (most cited first, 30 of 36)

- [FLASK: Fine-grained Language Model Evaluation based on Alignment Skill Sets](https://mlsanity.com/paper/2307.10928/) (2023)
- [CREAK: A Dataset for Commonsense Reasoning over Entity Knowledge](https://mlsanity.com/paper/2109.01653/) (2021)
- [FactKG: Fact Verification via Reasoning on Knowledge Graphs](https://mlsanity.com/paper/2305.06590/) (2023)
- [(QA)2: Question Answering with Questionable Assumptions](https://mlsanity.com/paper/2212.10003/) (2022)
- [Knowing What LLMs DO NOT Know: A Simple Yet Effective Self-Detection Method](https://mlsanity.com/paper/2310.17918/) (2023)
- [Missing Counter-Evidence Renders NLP Fact-Checking Unrealistic for Misinformation](https://mlsanity.com/paper/2210.13865/) (2022)
- [Stretching Sentence-pair NLI Models to Reason over Long Documents and Clusters](https://mlsanity.com/paper/2204.07447/) (2022)
- [Evidentiality-guided Generation for Knowledge-Intensive NLP Tasks](https://mlsanity.com/paper/2112.08688/) (2021)
- [Mind the Confidence Gap: Overconfidence, Calibration, and Distractor Effects in Large Language Models](https://mlsanity.com/paper/2502.11028/) (2025)
- [Language Models Hallucinate, but May Excel at Fact Verification](https://mlsanity.com/paper/2310.14564/) (2023)
- [SimRAG: Self-Improving Retrieval-Augmented Generation for Adapting Large Language Models to Specialized Domains](https://mlsanity.com/paper/2410.17952/) (2024)
- [FACTIFY-5WQA: 5W Aspect-based Fact Verification through Question Answering](https://mlsanity.com/paper/2305.04329/) (2023)
- [Claim Verification in the Age of Large Language Models: A Survey](https://mlsanity.com/paper/2408.14317/) (2024)
- [FACTIFY3M: A Benchmark for Multimodal Fact Verification with Explainability through 5W Question-Answering](https://mlsanity.com/paper/2306.05523/) (2023)
- [Web Retrieval Agents for Evidence-Based Misinformation Detection](https://mlsanity.com/paper/2409.00009/) (2024)
- [Claim-Dissector: An Interpretable Fact-Checking System with Joint Re-ranking and Veracity Prediction](https://mlsanity.com/paper/2207.14116/) (2022)
- [A Guide to Misinformation Detection Data and Evaluation](https://mlsanity.com/paper/2411.05060/) (2024)
- [Passage-Mask: A Learnable Regularization Strategy for Retriever-Reader Models](https://mlsanity.com/paper/2211.00915/) (2022)
- [The Battle of LLMs: A Comparative Study in Conversational QA Tasks](https://mlsanity.com/paper/2405.18344/) (2024)
- [Investigating Zero- and Few-shot Generalization in Fact Verification](https://mlsanity.com/paper/2309.09444/) (2023)
- [Fact or Fiction? Improving Fact Verification with Knowledge Graphs through Simplified Subgraph Retrievals](https://mlsanity.com/paper/2408.07453/) (2024)
- [ChatGPT-Crawler: Find out if ChatGPT really knows what it's talking about](https://mlsanity.com/paper/2304.03325/) (2023)
- [SituatedGen: Incorporating Geographical and Temporal Contexts into Generative Commonsense Reasoning](https://mlsanity.com/paper/2306.12552/) (2023)
- [EHRCon: Dataset for Checking Consistency between Unstructured Notes and Structured Tables in Electronic Health Records](https://mlsanity.com/paper/2406.16341/) (2024)
- [A Benchmark for Open-Domain Numerical Fact-Checking Enhanced by Claim Decomposition](https://mlsanity.com/paper/2510.22055/) (2025)
- [Tool-MAD: A Multi-Agent Debate Framework for Fact Verification with Diverse Tool Augmentation and Adaptive Retrieval](https://mlsanity.com/paper/2601.04742/) (2026)
- [QuestGen: Effectiveness of Question Generation Methods for Fact-Checking Applications](https://mlsanity.com/paper/2407.21441/) (2024)
- [Risks and NLP Design: A Case Study on Procedural Document QA](https://mlsanity.com/paper/2408.11860/) (2024)
- [Overview of Factify5WQA: Fact Verification through 5W Question-Answering](https://mlsanity.com/paper/2410.04236/) (2024)
- [PASS-FC: Progressive and Adaptive Search Scheme for Fact Checking of Comprehensive Claims](https://mlsanity.com/paper/2504.09866/) (2025)

## Abstract (arXiv)

Despite significant interest in developing general purpose fact checking models, it is challenging to construct a large-scale fact verification dataset with realistic real-world claims. Existing claims are either authored by crowdworkers, thereby introducing subtle biases that are difficult to control for, or manually verified by professional fact checkers, causing them to be expensive and limited in scale. In this paper, we construct a large-scale challenging fact verification dataset called FAVIQ, consisting of 188k claims derived from an existing corpus of ambiguous information-seeking questions. The ambiguities in the questions enable automatically constructing true and false claims that reflect user confusions (e.g., the year of the movie being filmed vs. being released). Claims in FAVIQ are verified to be natural, contain little lexical bias, and require a complete understanding of the evidence for verification. Our experiments show that the state-of-the-art models are far from solving our new task. Moreover, training on our data helps in professional fact-checking, outperforming models trained on the widely used dataset FEVER or in-domain data by up to 17% absolute. Altogether, our data will serve as a challenging benchmark for natural language understanding and support future progress in professional fact checking.