# FEVEROUS: Fact Extraction and VERification Over Unstructured and Structured information

arXiv: https://arxiv.org/abs/2106.05707 · HTML: https://mlsanity.com/paper/2106.05707/

## Key facts

- Cited by 167 papers in the map (data through 14 Oct 2026).
- Builds on 33 of its 69 references that are in the map.
- First posted on arXiv 10 Jun 2021.

## Builds on (most cited first, 30 of 33)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Semi-Supervised Classification with Graph Convolutional Networks](https://mlsanity.com/paper/1609.02907/) (2016)
- [A Broad-Coverage Challenge Corpus for Sentence Understanding through Inference](https://mlsanity.com/paper/1704.05426/) (2017)
- [A large annotated corpus for learning natural language inference](https://mlsanity.com/paper/1508.05326/) (2015)
- [FEVER: a large-scale dataset for Fact Extraction and VERification](https://mlsanity.com/paper/1803.05355/) (2018)
- [DROP: A Reading Comprehension Benchmark Requiring Discrete Reasoning Over Paragraphs](https://mlsanity.com/paper/1903.00161/) (2019)
- [Adversarial NLI: A New Benchmark for Natural Language Understanding](https://mlsanity.com/paper/1910.14599/) (2019)
- [Reading Wikipedia to Answer Open-Domain Questions](https://mlsanity.com/paper/1704.00051/) (2017)
- [The Mythos of Model Interpretability](https://mlsanity.com/paper/1606.03490/) (2016)
- [TabNet: Attentive Interpretable Tabular Learning](https://mlsanity.com/paper/1908.07442/) (2019)
- [Fact or Fiction: Verifying Scientific Claims](https://mlsanity.com/paper/2004.14974/) (2020)
- [TAPAS: Weakly Supervised Table Parsing via Pre-training](https://mlsanity.com/paper/2004.02349/) (2020)
- [TabFact: A Large-scale Dataset for Table-based Fact Verification](https://mlsanity.com/paper/1909.02164/) (2019)
- ["Liar, Liar Pants on Fire": A New Benchmark Dataset for Fake News Detection](https://mlsanity.com/paper/1705.00648/) (2017)
- [HybridQA: A Dataset of Multi-Hop Question Answering over Tabular and Textual Data](https://mlsanity.com/paper/2004.07347/) (2020)
- [Explainable Automated Fact-Checking for Public Health Claims](https://mlsanity.com/paper/2010.09926/) (2020)
- [HoVer: A Dataset for Many-Hop Fact Extraction And Claim Verification](https://mlsanity.com/paper/2011.03088/) (2020)
- [CLIMATE-FEVER: A Dataset for Verification of Real-World Climate Claims](https://mlsanity.com/paper/2012.00614/) (2020)
- [MultiFC: A Real-World Multi-Domain Dataset for Evidence-Based Fact Checking of Claims](https://mlsanity.com/paper/1909.03242/) (2019)
- [Open Question Answering over Tables and Text](https://mlsanity.com/paper/2010.10439/) (2020)
- [Automated Fact-Checking for Assisting Human Fact-Checkers](https://mlsanity.com/paper/2103.07769/) (2021)
- [Towards Debiasing Fact Verification Models](https://mlsanity.com/paper/1908.05267/) (2019)
- [Detection and Resolution of Rumours in Social Media: A Survey](https://mlsanity.com/paper/1704.00656/) (2017)
- [Automated Fact Checking: Task formulations, methods and future directions](https://mlsanity.com/paper/1806.07687/) (2018)
- [INFOTABS: Inference on Tables as Semi-structured Data](https://mlsanity.com/paper/2005.06117/) (2020)
- [Explainable Automated Fact-Checking: A Survey](https://mlsanity.com/paper/2011.03870/) (2020)
- [A Survey on Stance Detection for Mis- and Disinformation Identification](https://mlsanity.com/paper/2103.00242/) (2021)
- [A Richly Annotated Corpus for Different Tasks in Automated Fact-Checking](https://mlsanity.com/paper/1911.01214/) (2019)
- [The Limitations of Stylometry for Detecting Machine-Generated Fake News](https://mlsanity.com/paper/1908.09805/) (2019)
- [Integrating Stance Detection and Fact Checking in a Unified Corpus](https://mlsanity.com/paper/1804.08012/) (2018)

## Built on by (most cited first, 30 of 167)

- [TruthfulQA: Measuring How Models Mimic Human Falsehoods](https://mlsanity.com/paper/2109.07958/) (2021)
- [A Survey on Automated Fact-Checking](https://mlsanity.com/paper/2108.11896/) (2021)
- [Enhancing Retrieval-Augmented Large Language Models with Iterative Retrieval-Generation Synergy](https://mlsanity.com/paper/2305.15294/) (2023)
- [Large Language Models are Versatile Decomposers: Decompose Evidence and Questions for Table-based Reasoning](https://mlsanity.com/paper/2301.13808/) (2023)
- [Large Language Models(LLMs) on Tabular Data: Prediction, Generation, and Understanding -- A Survey](https://mlsanity.com/paper/2402.17944/) (2024)
- [Knowledge Graph Prompting for Multi-Document Question Answering](https://mlsanity.com/paper/2308.11730/) (2023)
- [Fact-Checking Complex Claims with Program-Guided Reasoning](https://mlsanity.com/paper/2305.12744/) (2023)
- [Table Meets LLM: Can Large Language Models Understand Structured Table Data? A Benchmark and Empirical Study](https://mlsanity.com/paper/2305.13062/) (2023)
- [Truthful AI: Developing and governing AI that does not lie](https://mlsanity.com/paper/2110.06674/) (2021)
- [End-to-End Multimodal Fact-Checking and Explanation Generation: A Challenging Dataset and Models](https://mlsanity.com/paper/2205.12487/) (2022)
- [Chain of Preference Optimization: Improving Chain-of-Thought Reasoning in LLMs](https://mlsanity.com/paper/2406.09136/) (2024)
- [Answering Questions by Meta-Reasoning over Multiple Chains of Thought](https://mlsanity.com/paper/2304.13007/) (2023)
- [Automatic Evaluation of Attribution by Large Language Models](https://mlsanity.com/paper/2305.06311/) (2023)
- [Generating Literal and Implied Subquestions to Fact-check Complex Claims](https://mlsanity.com/paper/2205.06938/) (2022)
- [Retrieval Augmented Generation (RAG) and Beyond: A Comprehensive Survey on How to Make your LLMs use External Data More Wisely](https://mlsanity.com/paper/2409.14924/) (2024)
- [DialFact: A Benchmark for Fact-Checking in Dialogue](https://mlsanity.com/paper/2110.08222/) (2021)
- [A Survey of LLM-based Deep Search Agents: Paradigm, Optimization, Evaluation, and Challenges](https://mlsanity.com/paper/2508.05668/) (2025)
- [FactKG: Fact Verification via Reasoning on Knowledge Graphs](https://mlsanity.com/paper/2305.06590/) (2023)
- [Automated Fact-Checking: A Survey](https://mlsanity.com/paper/2109.11427/) (2021)
- [ProoFVer: Natural Logic Theorem Proving for Fact Verification](https://mlsanity.com/paper/2108.11357/) (2021)
- [Multimodal Automated Fact-Checking: A Survey](https://mlsanity.com/paper/2305.13507/) (2023)
- [SCITAB: A Challenging Benchmark for Compositional Reasoning and Claim Verification on Scientific Tables](https://mlsanity.com/paper/2305.13186/) (2023)
- [Do Large Language Models Know about Facts?](https://mlsanity.com/paper/2310.05177/) (2023)
- [Missing Counter-Evidence Renders NLP Fact-Checking Unrealistic for Misinformation](https://mlsanity.com/paper/2210.13865/) (2022)
- [RobuT: A Systematic Study of Table QA Robustness Against Human-Annotated Adversarial Perturbations](https://mlsanity.com/paper/2306.14321/) (2023)
- [CHEF: A Pilot Chinese Dataset for Evidence-Based Fact-Checking](https://mlsanity.com/paper/2206.11863/) (2022)
- [Explainable Claim Verification via Knowledge-Grounded Reasoning with Large Language Models](https://mlsanity.com/paper/2310.05253/) (2023)
- [Reading and Reasoning over Chart Images for Evidence-based Automated Fact-Checking](https://mlsanity.com/paper/2301.11843/) (2023)
- [Zoom Out and Observe: News Environment Perception for Fake News Detection](https://mlsanity.com/paper/2203.10885/) (2022)
- [TAP4LLM: Table Provider on Sampling, Augmenting, and Packing Semi-structured Data for Large Language Model Reasoning](https://mlsanity.com/paper/2312.09039/) (2023)

## Abstract (arXiv)

Fact verification has attracted a lot of attention in the machine learning and natural language processing communities, as it is one of the key methods for detecting misinformation. Existing large-scale benchmarks for this task have focused mostly on textual sources, i.e. unstructured information, and thus ignored the wealth of information available in structured formats, such as tables. In this paper we introduce a novel dataset and benchmark, Fact Extraction and VERification Over Unstructured and Structured information (FEVEROUS), which consists of 87,026 verified claims. Each claim is annotated with evidence in the form of sentences and/or cells from tables in Wikipedia, as well as a label indicating whether this evidence supports, refutes, or does not provide enough information to reach a verdict. Furthermore, we detail our efforts to track and minimize the biases present in the dataset and could be exploited by models, e.g. being able to predict the label without using evidence. Finally, we develop a baseline for verifying claims against text and tables which predicts both the correct evidence and verdict for 18% of the claims.