# AVeriTeC: A Dataset for Real-world Claim Verification with Evidence from the Web

arXiv: https://arxiv.org/abs/2305.13117 · HTML: https://mlsanity.com/paper/2305.13117/

## Key facts

- Cited by 144 papers in the map (data through 14 Oct 2026).
- Builds on 35 of its 98 references that are in the map.
- First posted on arXiv 22 May 2023.

## Builds on (most cited first, 30 of 35)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension](https://mlsanity.com/paper/1910.13461/) (2019)
- [BLOOM: A 176B-Parameter Open-Access Multilingual Language Model](https://mlsanity.com/paper/2211.05100/) (2022)
- [FEVER: a large-scale dataset for Fact Extraction and VERification](https://mlsanity.com/paper/1803.05355/) (2018)
- [BLEURT: Learning Robust Metrics for Text Generation](https://mlsanity.com/paper/2004.04696/) (2020)
- [What Makes Good In-Context Examples for GPT-3?](https://mlsanity.com/paper/2101.06804/) (2021)
- [Learning To Retrieve Prompts for In-Context Learning](https://mlsanity.com/paper/2112.08633/) (2021)
- [Fact or Fiction: Verifying Scientific Claims](https://mlsanity.com/paper/2004.14974/) (2020)
- [A Survey on Automated Fact-Checking](https://mlsanity.com/paper/2108.11896/) (2021)
- ["Liar, Liar Pants on Fire": A New Benchmark Dataset for Fake News Detection](https://mlsanity.com/paper/1705.00648/) (2017)
- [Document Expansion by Query Prediction](https://mlsanity.com/paper/1904.08375/) (2019)
- [Get Your Vitamin C! Robust Fact Verification with Contrastive Evidence](https://mlsanity.com/paper/2103.08541/) (2021)
- [CLIMATE-FEVER: A Dataset for Verification of Real-World Climate Claims](https://mlsanity.com/paper/2012.00614/) (2020)
- [Generation-Augmented Retrieval for Open-domain Question Answering](https://mlsanity.com/paper/2009.08553/) (2020)
- [MultiFC: A Real-World Multi-Domain Dataset for Evidence-Based Fact Checking of Claims](https://mlsanity.com/paper/1909.03242/) (2019)
- [Automated Fact-Checking for Assisting Human Fact-Checkers](https://mlsanity.com/paper/2103.07769/) (2021)
- [COVID-Fact: Fact Extraction and Verification of Real-World Claims on COVID-19 Pandemic](https://mlsanity.com/paper/2106.03794/) (2021)
- [SciFact-Open: Towards open-domain scientific claim verification](https://mlsanity.com/paper/2210.13777/) (2022)
- [X-FACT: A New Benchmark Dataset for Multilingual Fact Checking](https://mlsanity.com/paper/2106.09248/) (2021)
- [Explainable Automated Fact-Checking: A Survey](https://mlsanity.com/paper/2011.03870/) (2020)
- [A Richly Annotated Corpus for Different Tasks in Automated Fact-Checking](https://mlsanity.com/paper/1911.01214/) (2019)
- [Generating Literal and Implied Subquestions to Fact-check Complex Claims](https://mlsanity.com/paper/2205.06938/) (2022)
- [Generating Fact Checking Briefs](https://mlsanity.com/paper/2011.05448/) (2020)
- [Missing Counter-Evidence Renders NLP Fact-Checking Unrealistic for Misinformation](https://mlsanity.com/paper/2210.13865/) (2022)
- [Claim Matching Beyond English to Scale Global Fact-Checking](https://mlsanity.com/paper/2106.00853/) (2021)
- [Multi-Hop Fact Checking of Political Claims](https://mlsanity.com/paper/2009.06401/) (2020)
- [CHEF: A Pilot Chinese Dataset for Evidence-Based Fact-Checking](https://mlsanity.com/paper/2206.11863/) (2022)
- [Investigating Reasons for Disagreement in Natural Language Inference](https://mlsanity.com/paper/2209.03392/) (2022)
- [Fully Automated Fact Checking Using External Sources](https://mlsanity.com/paper/1710.00341/) (2017)

## Built on by (most cited first, 30 of 144)

- [Knowledge Conflicts for LLMs: A Survey](https://mlsanity.com/paper/2403.08319/) (2024)
- [Language agents achieve superhuman synthesis of scientific knowledge](https://mlsanity.com/paper/2409.13740/) (2024)
- [VERISCORE: Evaluating the factuality of verifiable claims in long-form text generation](https://mlsanity.com/paper/2406.19276/) (2024)
- [DiscoveryBench: Towards Data-Driven Discovery with Large Language Models](https://mlsanity.com/paper/2407.01725/) (2024)
- [DEFAME: Dynamic Evidence-based FAct-checking with Multimodal Experts](https://mlsanity.com/paper/2412.10510/) (2024)
- [The Automated Verification of Textual Claims (AVeriTeC) Shared Task](https://mlsanity.com/paper/2410.23850/) (2024)
- [Show Me the Work: Fact-Checkers' Requirements for Explainable Automated Fact-Checking](https://mlsanity.com/paper/2502.09083/) (2025)
- [Worse than Zero-shot? A Fact-Checking Dataset for Evaluating the Robustness of RAG Against Misleading Retrievals](https://mlsanity.com/paper/2502.16101/) (2025)
- [Are More LLM Calls All You Need? Towards Scaling Laws of Compound Inference Systems](https://mlsanity.com/paper/2403.02419/) (2024)
- [Evidence-backed Fact Checking using RAG and Few-Shot In-Context Learning with LLMs](https://mlsanity.com/paper/2408.12060/) (2024)
- [Claim Verification in the Age of Large Language Models: A Survey](https://mlsanity.com/paper/2408.14317/) (2024)
- [QuanTemp: A real-world open-domain benchmark for fact-checking numerical claims](https://mlsanity.com/paper/2403.17169/) (2024)
- [Generative Large Language Models in Automated Fact-Checking: A Survey](https://mlsanity.com/paper/2407.02351/) (2024)
- [Ev2R: Evaluating Evidence Retrieval in Automated Fact-Checking](https://mlsanity.com/paper/2411.05375/) (2024)
- [AVerImaTeC: A Dataset for Automatic Verification of Image-Text Claims with Evidence from the Web](https://mlsanity.com/paper/2505.17978/) (2025)
- [Resolving Conflicting Evidence in Automated Fact-Checking: A Study on Retrieval-Augmented LLMs](https://mlsanity.com/paper/2505.17762/) (2025)
- [Do We Need Language-Specific Fact-Checking Models? The Case of Chinese](https://mlsanity.com/paper/2401.15498/) (2024)
- [Surprising Efficacy of Fine-Tuned Transformers for Fact-Checking over Larger Language Models](https://mlsanity.com/paper/2402.12147/) (2024)
- [Missci: Reconstructing Fallacies in Misrepresented Science](https://mlsanity.com/paper/2406.03181/) (2024)
- [COVE: COntext and VEracity prediction for out-of-context images](https://mlsanity.com/paper/2502.01194/) (2025)
- [The CLEF-2025 CheckThat! Lab: Subjectivity, Fact-Checking, Claim Normalization, and Retrieval](https://mlsanity.com/paper/2503.14828/) (2025)
- [Hallucination to Truth: A Review of Fact-Checking and Factuality Evaluation in Large Language Models](https://mlsanity.com/paper/2508.03860/) (2025)
- [Improving Health Question Answering with Reliable and Time-Aware Evidence Retrieval](https://mlsanity.com/paper/2404.08359/) (2024)
- [FactCheck Editor: Multilingual Text Editor with End-to-End fact-checking](https://mlsanity.com/paper/2404.19482/) (2024)
- [HerO at AVeriTeC: The Herd of Open Large Language Models for Verifying Real-World Claims](https://mlsanity.com/paper/2410.12377/) (2024)
- [CLAIMCHECK: How Grounded are LLM Critiques of Scientific Papers?](https://mlsanity.com/paper/2503.21717/) (2025)
- [SemEval-2025 Task 7: Multilingual and Crosslingual Fact-Checked Claim Retrieval](https://mlsanity.com/paper/2505.10740/) (2025)
- [Generating Media Background Checks for Automated Source Critical Reasoning](https://mlsanity.com/paper/2409.00781/) (2024)
- [FactLens: Benchmarking Fine-Grained Fact Verification](https://mlsanity.com/paper/2411.05980/) (2024)
- [Multimodal Fact-Checking with Vision Language Models: A Probing Classifier based Solution with Embedding Strategies](https://mlsanity.com/paper/2412.05155/) (2024)

## Abstract (arXiv)

Existing datasets for automated fact-checking have substantial limitations, such as relying on artificial claims, lacking annotations for evidence and intermediate reasoning, or including evidence published after the claim. In this paper we introduce AVeriTeC, a new dataset of 4,568 real-world claims covering fact-checks by 50 different organizations. Each claim is annotated with question-answer pairs supported by evidence available online, as well as textual justifications explaining how the evidence combines to produce a verdict. Through a multi-round annotation process, we avoid common pitfalls including context dependence, evidence insufficiency, and temporal leakage, and reach a substantial inter-annotator agreement of kappa=0.619 on verdicts. We develop a baseline as well as an evaluation scheme for verifying claims through several question-answering steps against the open web.