mlsanityData to 7 Oct 2026

Fool Me Twice: Entailment from Wikipedia Gamification

arXiv 2104.04725 ↗First posted 10 Apr 2021cs.CLcited by 50

Not on the map. Why?

Abstract (arXiv)

We release FoolMeTwice (FM2 for short), a large dataset of challenging entailment pairs collected through a fun multi-player game. Gamification encourages adversarial examples, drastically lowering the number of examples that can be solved using "shortcuts" compared to other popular entailment datasets. Players are presented with two tasks. The first task asks the player to write a plausible claim based on the evidence from a Wikipedia page. The second one shows two plausible claims written by other players, one of which is false, and the goal is to identify it before the time runs out. Players "pay" to see clues retrieved from the evidence pool: the more evidence the player needs, the harder the claim. Game-play between motivated players leads to diverse strategies for crafting claims, such as temporal inference and diverting to unrelated evidence, and results in higher quality data for the entailment and evidence retrieval tasks. We open source the dataset and the game code.

Key facts. Cited by 50 papers in the map (data through 7 Oct 2026). Builds on 23 of its 44 references that are in the map. First posted on arXiv 10 Apr 2021.

Builds on

23 of 44 references.

References · 23

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
  3. 2020Dense Passage Retrieval for Open-Domain Question Answeringcited by 3,779
  4. 2017A Broad-Coverage Challenge Corpus for Sentence Understanding through Inferencecited by 2,665
  5. 2020REALM: Retrieval-Augmented Language Model Pre-Trainingcited by 2,126
  6. 2015A large annotated corpus for learning natural language inferencecited by 1,837
  7. 2019Adversarial NLI: A New Benchmark for Natural Language Understandingcited by 798
  8. 2017Reading Wikipedia to Answer Open-Domain Questionscited by 777
  9. 2018Annotation Artifacts in Natural Language Inference Datacited by 656
  10. 2020KILT: a Benchmark for Knowledge Intensive Language Taskscited by 571
  11. 2018Hypothesis Only Baselines in Natural Language Inferencecited by 291
  12. 2019Don't Take the Easy Way Out: Ensemble Based Methods for Avoiding Known Dataset Biasescited by 279
11 more
  1. 2020HoVer: A Dataset for Many-Hop Fact Extraction And Claim Verificationcited by 199
  2. 2019Towards Debiasing Fact Verification Modelscited by 143
  3. 2020Beat the AI: Investigating Adversarial Human Annotation for Reading Comprehensioncited by 131
  4. 2020Towards Debiasing NLU Models from Unknown Biasescited by 116
  5. 2019Fine-grained Fact Verification with Kernel Graph Attention Networkcited by 105
  6. 2019GEAR: Graph-based Evidence Aggregating and Reasoning for Fact Verificationcited by 100
  7. 2020Understanding tables with intermediate pre-trainingcited by 68
  8. 2018Performance Impact Caused by Hidden Bias of Training Data for Recognizing Textual Entailmentcited by 58
  9. 2019What Question Answering can Learn from Trivia Nerdscited by 21
  10. 2020Asking Crowdworkers to Write Entailment Examples: The Best of Bad Optionscited by 8
  11. 2015Quizz: Targeted crowdsourcing with a billion (potential) userscited by 4

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 8; 2022: 6; 2023: 11; 2024: 10; 2025: 12; 2026: 3.

50 papers.

Sort:

By area:

  1. 2021A Survey on Automated Fact-Checkingcited by 376
  2. 2022Generate rather than Retrieve: Large Language Models are Strong Context Generatorscited by 296
  3. 2023Survey on Factuality in Large Language Models: Knowledge, Retrieval and Domain-Specificitycited by 204
  4. 2022CommonsenseQA 2.0: Exposing the Limits of AI through Gamificationcited by 108
  5. 2021CREAK: A Dataset for Commonsense Reasoning over Entity Knowledgecited by 88
  6. 2025A Survey of LLM-based Deep Search Agents: Paradigm, Optimization, Evaluation, and Challengescited by 58
  7. 2023Large Language Models Help Humans Verify Truthfulness -- Except When They Are Convincingly Wrongcited by 51
  8. 2023Do Large Language Models Know about Facts?cited by 49
  9. 2022Missing Counter-Evidence Renders NLP Fact-Checking Unrealistic for Misinformationcited by 45
  10. 2022Synthetic Disinformation Attacks on Automated Fact Verification Systemscited by 44
  11. 2021Mention Memory: incorporating textual knowledge into Transformers through entity mention attentioncited by 42
  12. 2024When Large Language Models Meet Vector Databases: A Surveycited by 39
38 more
  1. 2023Augmented Large Language Models with Parametric Knowledge Guidingcited by 37
  2. 2021FaVIQ: FAct Verification from Information-seeking Questionscited by 36
  3. 2023EX-FEVER: A Dataset for Multi-hop Explainable Fact Verificationcited by 36
  4. 2023Language Models Hallucinate, but May Excel at Fact Verificationcited by 35
  5. 2023Trends in Integration of Knowledge and Large Language Models: A Survey and Taxonomy of Methods, Benchmarks, and Applicationscited by 28
  6. 2022Varifocal Question Generation for Fact-checkingcited by 27
  7. 2021Analyzing Dynamic Adversarial Training Data in the Limitcited by 24
  8. 2021CrossAug: A Contrastive Data Augmentation Method for Debiasing Fact Verification Modelscited by 23
  9. 2024Claim Verification in the Age of Large Language Models: A Surveycited by 23
  10. 2022WinoGAViL: Gamified Association Benchmark to Challenge Vision-and-Language Modelscited by 20
  11. 2025MCTS-RAG: Enhancing Retrieval-Augmented Generation with Monte Carlo Tree Searchcited by 20
  12. 2024Generative Large Language Models in Automated Fact-Checking: A Surveycited by 19
  13. 2021Explainable Fact-checking through Question Answeringcited by 16
  14. 2025Hallucination to Truth: A Review of Fact-Checking and Factuality Evaluation in Large Language Modelscited by 13
  15. 2023Continually Improving Extractive QA via Human Feedbackcited by 11
  16. 2023EXPLAIN, EDIT, GENERATE: Rationale-Sensitive Counterfactual Data Augmentation for Multi-hop Fact Verificationcited by 9
  17. 2024More Victories, Less Cooperation: Assessing Cicero's Diplomacy Playcited by 9
  18. 2024On Evaluating Explanation Utility for Human-AI Decision Making in NLPcited by 9
  19. 2025InfoGain-RAG: Boosting Retrieval-Augmented Generation via Document Information Gain-based Reranking and Filteringcited by 8
  20. 2024Is your benchmark truly adversarial? AdvScore: Evaluating Human-Grounded Adversarialnesscited by 7
  21. 2024SMART-RAG: Selection using Determinantal Matrices for Augmented Retrievalcited by 6
  22. 2025Domain Regeneration: How well do LLMs match syntactic properties of text domains?cited by 6
  23. 2023Not all Fake News is Written: A Dataset and Analysis of Misleading Video Headlinescited by 5
  24. 2025GRACE: A Granular Benchmark for Evaluating Model Calibration against Human Calibrationcited by 5
  25. 2025Adversarial Attacks Against Automated Fact-Checking: A Surveycited by 5
  26. 2023Investigating Zero- and Few-shot Generalization in Fact Verificationcited by 4
  27. 2025Facts are Harder Than Opinions -- A Multilingual, Comparative Analysis of LLM-Based Fact-Checking Reliabilitycited by 3
  28. 2025Can Performant LLMs Be Ethical? Quantifying the Impact of Web Crawling Opt-Outscited by 2
  29. 2025SUCEA: Reasoning-Intensive Retrieval for Adversarial Fact-checking through Claim Decomposition and Editingcited by 2
  30. 2025Discrepancy Detection at the Data Level: Toward Consistent Multilingual Question Answeringcited by 2
  31. 2026DecomposeRL: Learning to Ask Useful, Informative, and Diverse Questions for Semi-Supervised, Traceable Claim Verificationcited by 2
  32. 2021The Choice of Knowledge Base in Automated Claim Checkingcited by 1
  33. 2024How the Advent of Ubiquitous Large Language Models both Stymie and Turbocharge Dynamic Adversarial Question Generationcited by 1
  34. 2025Self-Rationalization in the Wild: A Large Scale Out-of-Distribution Evaluation on NLI-related taskscited by 1
  35. 2024Zero-shot Factual Consistency Evaluation Across Domainscited by 0
  36. 2024Gamified crowd-sourcing of high-quality data for visual fine-tuningcited by 0
  37. 2026AtomEval: Validity-Aware Atomic Evaluation of Adversarial Claim Rewriting in Fact Verificationcited by 0
  38. 2026It Matters How You Say It: Exploring Rhetorical Patterns for AI-Assisted Information Evaluationcited by 0