mlsanityData to 14 Oct 2026

AVeriTeC: A Dataset for Real-world Claim Verification with Evidence from the Web

arXiv 2305.13117 ↗First posted 22 May 2023cs.CLcited by 144

Not on the map. Why?

Abstract (arXiv)

Existing datasets for automated fact-checking have substantial limitations, such as relying on artificial claims, lacking annotations for evidence and intermediate reasoning, or including evidence published after the claim. In this paper we introduce AVeriTeC, a new dataset of 4,568 real-world claims covering fact-checks by 50 different organizations. Each claim is annotated with question-answer pairs supported by evidence available online, as well as textual justifications explaining how the evidence combines to produce a verdict. Through a multi-round annotation process, we avoid common pitfalls including context dependence, evidence insufficiency, and temporal leakage, and reach a substantial inter-annotator agreement of kappa=0.619 on verdicts. We develop a baseline as well as an evaluation scheme for verifying claims through several question-answering steps against the open web.

Key facts. Cited by 144 papers in the map (data through 14 Oct 2026). Builds on 35 of its 98 references that are in the map. First posted on arXiv 22 May 2023.

Builds on

35 of 98 references.

References · 35

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  2. 2014Adam: A Method for Stochastic Optimizationcited by 37,240
  3. 2019BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehensioncited by 5,893
  4. 2022BLOOM: A 176B-Parameter Open-Access Multilingual Language Modelcited by 1,984
  5. 2018FEVER: a large-scale dataset for Fact Extraction and VERificationcited by 1,375
  6. 2020BLEURT: Learning Robust Metrics for Text Generationcited by 1,203
  7. 2021What Makes Good In-Context Examples for GPT-3?cited by 1,195
  8. 2021Learning To Retrieve Prompts for In-Context Learningcited by 648
  9. 2020Fact or Fiction: Verifying Scientific Claimscited by 554
  10. 2021A Survey on Automated Fact-Checkingcited by 378
  11. 2017"Liar, Liar Pants on Fire": A New Benchmark Dataset for Fake News Detectioncited by 363
  12. 2019Document Expansion by Query Predictioncited by 231
23 more
  1. 2021Get Your Vitamin C! Robust Fact Verification with Contrastive Evidencecited by 225
  2. 2020CLIMATE-FEVER: A Dataset for Verification of Real-World Climate Claimscited by 196
  3. 2020Generation-Augmented Retrieval for Open-domain Question Answeringcited by 186
  4. 2019MultiFC: A Real-World Multi-Domain Dataset for Evidence-Based Fact Checking of Claimscited by 169
  5. 2021Automated Fact-Checking for Assisting Human Fact-Checkerscited by 144
  6. 2021COVID-Fact: Fact Extraction and Verification of Real-World Claims on COVID-19 Pandemiccited by 100
  7. 2022SciFact-Open: Towards open-domain scientific claim verificationcited by 94
  8. 2021X-FACT: A New Benchmark Dataset for Multilingual Fact Checkingcited by 91
  9. 2020Explainable Automated Fact-Checking: A Surveycited by 83
  10. 2019A Richly Annotated Corpus for Different Tasks in Automated Fact-Checkingcited by 74
  11. 2022Generating Literal and Implied Subquestions to Fact-check Complex Claimscited by 66
  12. 2020Generating Fact Checking Briefscited by 50
  13. 2022Missing Counter-Evidence Renders NLP Fact-Checking Unrealistic for Misinformationcited by 45
  14. 2021Claim Matching Beyond English to Scale Global Fact-Checkingcited by 42
  15. 2020Multi-Hop Fact Checking of Political Claimscited by 37
  16. 2022CHEF: A Pilot Chinese Dataset for Evidence-Based Fact-Checkingcited by 37
  17. 2022Investigating Reasons for Disagreement in Natural Language Inferencecited by 29
  18. 2017Fully Automated Fact Checking Using External Sourcescited by 27
  19. 2022Varifocal Question Generation for Fact-checkingcited by 27
  20. 2023The Intended Uses of Automated Fact-Checking Artefacts: Why, How and Whocited by 19
  21. 2021Explainable Fact-checking through Question Answeringcited by 16
  22. 2019Automated Fact Checking in the News Roomcited by 13
  23. 2021The Case for Claim Difficulty Assessment in Automatic Fact Checkingcited by 5

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 1; 2024: 38; 2025: 45; 2026: 60.

144 papers.

Sort:

By area:

  1. 2024Knowledge Conflicts for LLMs: A Surveycited by 283
  2. 2024Language agents achieve superhuman synthesis of scientific knowledgecited by 137
  3. 2024VERISCORE: Evaluating the factuality of verifiable claims in long-form text generationcited by 117
  4. 2024DiscoveryBench: Towards Data-Driven Discovery with Large Language Modelscited by 109
  5. 2024DEFAME: Dynamic Evidence-based FAct-checking with Multimodal Expertscited by 43
  6. 2024The Automated Verification of Textual Claims (AVeriTeC) Shared Taskcited by 29
  7. 2025Show Me the Work: Fact-Checkers' Requirements for Explainable Automated Fact-Checkingcited by 29
  8. 2025Worse than Zero-shot? A Fact-Checking Dataset for Evaluating the Robustness of RAG Against Misleading Retrievalscited by 28
  9. 2024Are More LLM Calls All You Need? Towards Scaling Laws of Compound Inference Systemscited by 25
  10. 2024Evidence-backed Fact Checking using RAG and Few-Shot In-Context Learning with LLMscited by 24
  11. 2024Claim Verification in the Age of Large Language Models: A Surveycited by 24
  12. 2024QuanTemp: A real-world open-domain benchmark for fact-checking numerical claimscited by 23
38 more
  1. 2024Generative Large Language Models in Automated Fact-Checking: A Surveycited by 19
  2. 2024Ev2R: Evaluating Evidence Retrieval in Automated Fact-Checkingcited by 18
  3. 2025AVerImaTeC: A Dataset for Automatic Verification of Image-Text Claims with Evidence from the Webcited by 18
  4. 2025Resolving Conflicting Evidence in Automated Fact-Checking: A Study on Retrieval-Augmented LLMscited by 16
  5. 2024Do We Need Language-Specific Fact-Checking Models? The Case of Chinesecited by 15
  6. 2024Surprising Efficacy of Fine-Tuned Transformers for Fact-Checking over Larger Language Modelscited by 14
  7. 2024Missci: Reconstructing Fallacies in Misrepresented Sciencecited by 14
  8. 2025COVE: COntext and VEracity prediction for out-of-context imagescited by 14
  9. 2025The CLEF-2025 CheckThat! Lab: Subjectivity, Fact-Checking, Claim Normalization, and Retrievalcited by 14
  10. 2025Hallucination to Truth: A Review of Fact-Checking and Factuality Evaluation in Large Language Modelscited by 13
  11. 2024Improving Health Question Answering with Reliable and Time-Aware Evidence Retrievalcited by 12
  12. 2024FactCheck Editor: Multilingual Text Editor with End-to-End fact-checkingcited by 12
  13. 2024HerO at AVeriTeC: The Herd of Open Large Language Models for Verifying Real-World Claimscited by 12
  14. 2025CLAIMCHECK: How Grounded are LLM Critiques of Scientific Papers?cited by 12
  15. 2025SemEval-2025 Task 7: Multilingual and Crosslingual Fact-Checked Claim Retrievalcited by 11
  16. 2024Generating Media Background Checks for Automated Source Critical Reasoningcited by 10
  17. 2024FactLens: Benchmarking Fine-Grained Fact Verificationcited by 10
  18. 2024Multimodal Fact-Checking with Vision Language Models: A Probing Classifier based Solution with Embedding Strategiescited by 10
  19. 2024Evaluating Robustness of Generative Search Engine on Adversarial Factual Questionscited by 9
  20. 2024Revealing the Parametric Knowledge of Language Models: A Unified Framework for Attribution Methodscited by 9
  21. 2024A Reality Check on Context Utilisation for Retrieval-Augmented Generationcited by 9
  22. 2024Credible, Unreliable or Leaked?: Evidence Verification for Enhanced Automated Fact-checkingcited by 8
  23. 2024AIC CTU system at AVeriTeC: Re-framing automated fact-checking as a simple RAG taskcited by 8
  24. 2025FaStfact: Faster, Stronger Long-Form Factuality Evaluations in LLMscited by 8
  25. 2025The Missing Parts: Augmenting Fact Verification with Half-Truth Detectioncited by 7
  26. 2025M4FC: a Multimodal, Multilingual, Multicultural, Multitask Real-World Fact-Checking Datasetcited by 7
  27. 2024Document-level Claim Extraction and Decontextualisation for Fact-Checkingcited by 6
  28. 2024Contrastive Learning to Improve Retrieval for Real-world Fact Checkingcited by 6
  29. 2025The Next Phase of Scientific Fact-Checking: Advanced Evidence Retrieval from Complex Structured Academic Paperscited by 6
  30. 2025Debating Truth: Debate-driven Claim Verification with Multiple Large Language Model Agentscited by 6
  31. 2026VeriTaS: The First Dynamic Benchmark for Multimodal Automated Fact-Checkingcited by 6
  32. 2026Breaking the Chain: A Causal Analysis of LLM Faithfulness to Intermediate Structurescited by 6
  33. 2026Where Do Deep-Research Agents Go Wrong? Span-Level Error Localization in Agent Trajectoriescited by 6
  34. 2024LiveFC: A System for Live Fact-Checking of Audio Streamscited by 5
  35. 2025DRAG: Distilling RAG for SLMs from LLMs to Transfer Knowledge and Mitigate Hallucination via Evidence and Graph-based Distillationcited by 5
  36. 2025Adversarial Attacks Against Automated Fact-Checking: A Surveycited by 5
  37. 2026How Adversarial Environments Mislead Agentic AI?cited by 5
  38. 2023Give Me More Details: Improving Fact-Checking with Latent Retrievalcited by 4

50 most cited of 144 All 144 (JSON)