# A Guide to Misinformation Detection Data and Evaluation

arXiv: https://arxiv.org/abs/2411.05060 · HTML: https://mlsanity.com/paper/2411.05060/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- Builds on 41 of its 105 references that are in the map.
- First posted on arXiv 7 Nov 2024.

## Builds on (most cited first, 30 of 41)

- [Constitutional AI: Harmlessness from AI Feedback](https://mlsanity.com/paper/2212.08073/) (2022)
- [Shortcut Learning in Deep Neural Networks](https://mlsanity.com/paper/2004.07780/) (2020)
- [Datasheets for Datasets](https://mlsanity.com/paper/1803.09010/) (2018)
- [FEVER: a large-scale dataset for Fact Extraction and VERification](https://mlsanity.com/paper/1803.05355/) (2018)
- ["Liar, Liar Pants on Fire": A New Benchmark Dataset for Fake News Detection](https://mlsanity.com/paper/1705.00648/) (2017)
- [Fake News Detection on Social Media: A Data Mining Perspective](https://mlsanity.com/paper/1708.01967/) (2017)
- [Principle-Driven Self-Alignment of Language Models from Scratch with Minimal Human Supervision](https://mlsanity.com/paper/2305.03047/) (2023)
- [MEGA: Multilingual Evaluation of Generative AI](https://mlsanity.com/paper/2303.12528/) (2023)
- [FakeNewsNet: A Data Repository with News Content, Social Context and Spatialtemporal Information for Studying Fake News on Social Media](https://mlsanity.com/paper/1809.01286/) (2018)
- [HoVer: A Dataset for Many-Hop Fact Extraction And Claim Verification](https://mlsanity.com/paper/2011.03088/) (2020)
- [CLIMATE-FEVER: A Dataset for Verification of Real-World Climate Claims](https://mlsanity.com/paper/2012.00614/) (2020)
- [Long-form factuality in large language models](https://mlsanity.com/paper/2403.18802/) (2024)
- [Automatic Detection of Fake News](https://mlsanity.com/paper/1708.07104/) (2017)
- [COVID-Fact: Fact Extraction and Verification of Real-World Claims on COVID-19 Pandemic](https://mlsanity.com/paper/2106.03794/) (2021)
- [The science of fake news](https://mlsanity.com/paper/2307.07903/) (2023)
- [Towards LLM-based Fact Verification on News Claims with a Hierarchical Step-by-Step Prompting Method](https://mlsanity.com/paper/2310.00305/) (2023)
- [Combating Misinformation in the Age of LLMs: Opportunities and Challenges](https://mlsanity.com/paper/2311.05656/) (2023)
- [X-FACT: A New Benchmark Dataset for Multilingual Fact Checking](https://mlsanity.com/paper/2106.09248/) (2021)
- [CoAID: COVID-19 Healthcare Misinformation Dataset](https://mlsanity.com/paper/2006.00885/) (2020)
- [Fighting the COVID-19 Infodemic: Modeling the Perspective of Journalists, Fact-Checkers, Social Media Platforms, Policy Makers, and the Society](https://mlsanity.com/paper/2005.00033/) (2020)
- [A Survey on Natural Language Processing for Fake News Detection](https://mlsanity.com/paper/1811.00770/) (2018)
- [FakeCovid -- A Multilingual Cross-domain Fact Check News Dataset for COVID-19](https://mlsanity.com/paper/2006.11343/) (2020)
- [This Just In: Fake News Packs a Lot in Title, Uses Simpler, Repetitive Content in Text Body, More Similar to Satire than Real News](https://mlsanity.com/paper/1703.09398/) (2017)
- [Towards Reliable Misinformation Mitigation: Generalization, Uncertainty, and GPT-4](https://mlsanity.com/paper/2305.14928/) (2023)
- [BanFakeNews: A Dataset for Detecting Fake News in Bangla](https://mlsanity.com/paper/2004.08789/) (2020)
- [VERITE: A Robust Benchmark for Multimodal Misinformation Detection Accounting for Unimodal Bias](https://mlsanity.com/paper/2304.14133/) (2023)
- [MM-COVID: A Multilingual and Multimodal Data Repository for Combating COVID-19 Disinformation](https://mlsanity.com/paper/2011.04088/) (2020)
- [FaVIQ: FAct Verification from Information-seeking Questions](https://mlsanity.com/paper/2107.02153/) (2021)
- [Open, Closed, or Small Language Models for Text Classification?](https://mlsanity.com/paper/2308.10092/) (2023)
- [Combating Fake News: A Survey on Identification and Mitigation Techniques](https://mlsanity.com/paper/1901.06437/) (2019)

## Built on by (most cited first, 10 of 10)

- [TRUST-VL: An Explainable News Assistant for General Multimodal Misinformation Detection](https://mlsanity.com/paper/2509.04448/) (2025)
- [DeepFact: Co-Evolving Benchmarks and Agents for Deep Research Factuality](https://mlsanity.com/paper/2603.05912/) (2026)
- [MPCG: Multi-Round Persona-Conditioned Generation for Modeling the Evolution of Misinformation with LLMs](https://mlsanity.com/paper/2509.16564/) (2025)
- [All That Glisters Is Not Gold: A Benchmark for Reference-Free Counterfactual Financial Misinformation Detection](https://mlsanity.com/paper/2601.04160/) (2026)
- [BLUFF: Benchmarking the Detection of False and Synthetic Content across 58 Low-Resource Languages](https://mlsanity.com/paper/2603.00634/) (2026)
- [R2VC: Modular Fact-Checking with Retrieval, Verification, and Confidence Calibration](https://mlsanity.com/paper/2609.11955/) (2026)
- [From Veracity to Diffusion: Adressing Operational Challenges in Moving From Fake-News Detection to Information Disorders](https://mlsanity.com/paper/2512.02552/) (2025)
- [Benchmark Health Index: A Systematic Framework for Benchmarking the Benchmarks of LLMs](https://mlsanity.com/paper/2602.11674/) (2026)
- [IPO-Mine: A Toolkit and Dataset for Section-Structured Analysis of Long, Multimodal IPO Documents](https://mlsanity.com/paper/2605.28714/) (2026)
- [SFBench: The SciFy Scientific Feasibility Benchmark](https://mlsanity.com/paper/2606.29630/) (2026)

## Abstract (arXiv)

Misinformation is a complex societal issue, and mitigating solutions are difficult to create due to data deficiencies. To address this, we have curated the largest collection of (mis)information datasets in the literature, totaling 75. From these, we evaluated the quality of 36 datasets that consist of statements or claims, as well as the 9 datasets that consist of data in purely paragraph form. We assess these datasets to identify those with solid foundations for empirical work and those with flaws that could result in misleading and non-generalizable results, such as spurious correlations, or examples that are ambiguous or otherwise impossible to assess for veracity. We find the latter issue is particularly severe and affects most datasets in the literature. We further provide state-of-the-art baselines on all these datasets, but show that regardless of label quality, categorical labels may no longer give an accurate evaluation of detection model performance. Finally, we propose and highlight Evaluation Quality Assurance (EQA) as a tool to guide the field toward systemic solutions rather than inadvertently propagating issues in evaluation. Overall, this guide aims to provide a roadmap for higher quality data and better grounded evaluations, ultimately improving research in misinformation detection. All datasets and other artifacts are available at https://misinfo-datasets.complexdatalab.com/.