mlsanityData to 14 Oct 2026

Beat the AI: Investigating Adversarial Human Annotation for Reading Comprehension

arXiv 2002.00293 ↗First posted 2 Feb 2020cs.CLcited by 131

Not on the map. Why?

Abstract (arXiv)

Innovations in annotation methodology have been a catalyst for Reading Comprehension (RC) datasets and models. One recent trend to challenge current RC models is to involve a model in the annotation process: humans create questions adversarially, such that the model fails to answer them correctly. In this work we investigate this annotation methodology and apply it in three different settings, collecting a total of 36,000 samples with progressively stronger models in the annotation loop. This allows us to explore questions such as the reproducibility of the adversarial effect, transfer from data collected with varying model-in-the-loop strengths, and generalisation to data collected without a model. We find that training on adversarially collected samples leads to strong generalisation to non-adversarially collected datasets, yet with progressive performance deterioration with increasingly stronger models-in-the-loop. Furthermore, we find that stronger models can still learn from datasets collected with substantially weaker models-in-the-loop. When trained on data collected with a BiDAF model in the loop, RoBERTa achieves 39.9F1 on questions that it cannot answer when trained on SQuAD - only marginally lower than when trained on data collected using RoBERTa itself (41.0F1).

Key facts. Cited by 131 papers in the map (data through 14 Oct 2026). First posted on arXiv 2 Feb 2020.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2021: 29; 2022: 27; 2023: 32; 2024: 20; 2025: 15; 2026: 8.

131 papers.

Sort:

By area:

  1. 2022Super-NaturalInstructions: Generalization via Declarative Instructions on 1600+ NLP Taskscited by 868
  2. 2022Discovering Language Model Behaviors with Model-Written Evaluationscited by 836
  3. 2021Did Aristotle Use a Laptop? A Question Answering Benchmark with Implicit Reasoning Strategiescited by 743
  4. 2021Dynabench: Rethinking Benchmarking in NLPcited by 459
  5. 2024On LLMs-Driven Synthetic Data Generation, Curation, and Evaluation: A Surveycited by 244
  6. 2021Adversarial GLUE: A Multi-Task Benchmark for Robustness Evaluation of Language Modelscited by 232
  7. 2021A Survey of Human-in-the-loop for Machine Learningcited by 215
  8. 2022ZeroGen: Efficient Zero-shot Learning via Dataset Generationcited by 211
  9. 2021QuALITY: Question Answering with Long Input Texts, Yes!cited by 208
  10. 2022WANLI: Worker and AI Collaboration for Natural Language Inference Dataset Creationcited by 202
  11. 2024Aya Dataset: An Open-Access Collection for Multilingual Instruction Tuningcited by 177
  12. 2022Rich Knowledge Sources Bring Complex Knowledge Conflicts: Recalibrating Models to Reflect Conflicting Evidencecited by 144
38 more
  1. 2021CrossFit: A Few-shot Learning Challenge for Cross-task Generalization in NLPcited by 142
  2. 2022CommonsenseQA 2.0: Exposing the Limits of AI through Gamificationcited by 109
  3. 2021Measure and Improve Robustness in NLP Models: A Surveycited by 107
  4. 2022State-of-the-art generalisation research in NLP: A taxonomy and reviewcited by 98
  5. 2023Revisiting Out-of-distribution Robustness in NLP: Benchmark, Analysis, and LLMs Evaluationscited by 97
  6. 2022Evaluating Human-Language Model Interactioncited by 92
  7. 2022Parallel Context Windows for Large Language Modelscited by 81
  8. 2023Exploring the Benefits of Training Expert Language Models over Instruction Tuningcited by 80
  9. 2021Improving Question Answering Model Robustness with Synthetic Adversarial Data Generationcited by 79
  10. 2021Adversarial VQA: A New Benchmark for Evaluating the Robustness of VQA Modelscited by 75
  11. 2022Evaluating the Susceptibility of Pre-Trained Language Models via Handcrafted Adversarial Examplescited by 69
  12. 2023Ignore This Title and HackAPrompt: Exposing Systemic Vulnerabilities of LLMs through a Global Scale Prompt Hacking Competitioncited by 68
  13. 2023Elo Uncovered: Robustness and Best Practices in Language Model Evaluationcited by 61
  14. 2023Training Transformers with 4-bit Integerscited by 60
  15. 2021Human-Adversarial Visual Question Answeringcited by 57
  16. 2022UnifiedQA-v2: Stronger Generalization via Broader Cross-Format Trainingcited by 57
  17. 2021Dynaboard: An Evaluation-As-A-Service Platform for Holistic Next-Generation Benchmarkingcited by 53
  18. 2021NeurIPS 2020 EfficientQA Competition: Systems, Analyses and Lessons Learnedcited by 52
  19. 2021Fool Me Twice: Entailment from Wikipedia Gamificationcited by 50
  20. 2021Hatemoji: A Test Suite and Adversarially-Generated Dataset for Benchmarking and Detecting Emoji-based Hatecited by 50
  21. 2023RobuT: A Systematic Study of Table QA Robustness Against Human-Annotated Adversarial Perturbationscited by 43
  22. 2021COM2SENSE: A Commonsense Reasoning Benchmark with Complementary Sentencescited by 40
  23. 2021Why Machine Reading Comprehension Models Learn Shortcuts?cited by 40
  24. 2021The Dangers of Underclaiming: Reasons for Caution When Reporting How NLP Systems Failcited by 38
  25. 2024Generative AI for Synthetic Data Generation: Methods, Challenges and the Futurecited by 33
  26. 2021On the Efficacy of Adversarial Data Collection for Question Answering: Results from a Large-Scale Randomized Studycited by 31
  27. 2021Models in the Loop: Aiding Crowdworkers with Generative Annotation Assistantscited by 31
  28. 2023Mind the instructions: a holistic evaluation of consistency and interactions in prompt-based learningcited by 31
  29. 2023Assessing Language Model Deployment with Risk Cardscited by 30
  30. 2023Let's Synthesize Step by Step: Iterative Dataset Synthesis with Large Language Models by Extrapolating Errors from Small Modelscited by 30
  31. 2021Retrieval-guided Counterfactual Generation for QAcited by 29
  32. 2021Exploring Universal Intrinsic Task Subspace via Prompt Tuningcited by 25
  33. 2021Analyzing Dynamic Adversarial Training Data in the Limitcited by 24
  34. 2022CORE: A Retrieve-then-Edit Framework for Counterfactual Data Generationcited by 24
  35. 2021What Ingredients Make for an Effective Crowdsourcing Protocol for Difficult NLU Data Collection Tasks?cited by 23
  36. 2022WinoGAViL: Gamified Association Benchmark to Challenge Vision-and-Language Modelscited by 20
  37. 2023ScatterShot: Interactive In-context Example Curation for Text Transformationcited by 20
  38. 2024Adversarial Nibbler: An Open Red-Teaming Method for Identifying Diverse Harms in Text-to-Image Generationcited by 20

50 most cited of 131 All 131 (JSON)