mlsanityData to 7 Oct 2026

Improving Question Answering Model Robustness with Synthetic Adversarial Data Generation

arXiv 2104.08678 ↗First posted 18 Apr 2021cs.CL cs.LGcited by 77

Not on the map. Why?

Abstract (arXiv)

Despite recent progress, state-of-the-art question answering models remain vulnerable to a variety of adversarial attacks. While dynamic adversarial data collection, in which a human annotator tries to write examples that fool a model-in-the-loop, can improve model robustness, this process is expensive which limits the scale of the collected data. In this work, we are the first to use synthetic adversarial data generation to make question answering models more robust to human adversaries. We develop a data generation pipeline that selects source passages, identifies candidate answers, generates questions, then finally filters or re-labels them to improve quality. Using this approach, we amplify a smaller human-written adversarial dataset to a much larger set of synthetic question-answer pairs. By incorporating our synthetic data, we improve the state-of-the-art on the AdversarialQA dataset by 3.7F1 and improve model generalisation on nine of the twelve MRQA datasets. We further conduct a novel human-in-the-loop evaluation to show that our models are considerably more robust to new human-written adversarial examples: crowdworkers can fool our model only 8.8% of the time on average, compared to 17.6% for a model trained without synthetic data.

Key facts. Cited by 77 papers in the map (data through 7 Oct 2026). Builds on 32 of its 51 references that are in the map. First posted on arXiv 18 Apr 2021.

Builds on

32 of 51 references.

References · 32

  1. 2017Attention Is All You Needcited by 59,932
  2. 2019BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehensioncited by 5,874
  3. 2016SQuAD: 100,000+ Questions for Machine Comprehension of Textcited by 4,115
  4. 2014CIDEr: Consensus-based Image Description Evaluationcited by 2,027
  5. 2015A large annotated corpus for learning natural language inferencecited by 1,837
  6. 2017Understanding Black-box Predictions via Influence Functionscited by 1,714
  7. 2018Know What You Don't Know: Unanswerable Questions for SQuADcited by 1,538
  8. 2020Beyond Accuracy: Behavioral Testing of NLP models with CheckListcited by 979
  9. 2019DROP: A Reading Comprehension Benchmark Requiring Discrete Reasoning Over Paragraphscited by 883
  10. 2019Right for the Wrong Reasons: Diagnosing Syntactic Heuristics in Natural Language Inferencecited by 864
  11. 2019Adversarial NLI: A New Benchmark for Natural Language Understandingcited by 798
  12. 2018Annotation Artifacts in Natural Language Inference Datacited by 656
20 more
  1. 2017Adversarial Examples for Evaluating Reading Comprehension Systemscited by 568
  2. 2021Dynabench: Rethinking Benchmarking in NLPcited by 447
  3. 2016How NOT To Evaluate Your Dialogue System: An Empirical Study of Unsupervised Evaluation Metrics for Dialogue Response Generationcited by 356
  4. 2020Learning from the Worst: Dynamically Generated Datasets to Improve Online Hate Detectioncited by 217
  5. 2021PAQ: 65 Million Probably-Asked Questions and What You Can Do With Themcited by 198
  6. 2017Learning to Ask: Neural Question Generation for Reading Comprehensioncited by 186
  7. 2019MRQA 2019 Shared Task: Evaluating Generalization in Reading Comprehensioncited by 183
  8. 2019Are We Modeling the Task or the Annotator? An Investigation of Annotator Bias in Natural Language Understanding Datasetscited by 173
  9. 2014SimLex-999: Evaluating Semantic Models with (Genuine) Similarity Estimationcited by 170
  10. 2020Understanding Self-Training for Gradual Domain Adaptationcited by 149
  11. 2019Synthetic QA Corpora Generation with Roundtrip Consistencycited by 143
  12. 2020Question and Answer Test-Train Overlap in Open-Domain Question Answering Datasetscited by 137
  13. 2020Beat the AI: Investigating Adversarial Human Annotation for Reading Comprehensioncited by 131
  14. 2020Self-training Improves Pre-training for Natural Language Understandingcited by 96
  15. 2020Training Question Answering Models From Synthetic Datacited by 95
  16. 2017Neural Question Generation from Text: A Preliminary Studycited by 78
  17. 2020DynaSent: A Dynamic Benchmark for Sentiment Analysiscited by 63
  18. 2019Unsupervised Question Answering by Cloze Translationcited by 59
  19. 2018Harvesting Paragraph-Level Question-Answer Pairs from Wikipediacited by 49
  20. 2020Accelerating Real-Time Question Answering via Question Generationcited by 11

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 4; 2022: 24; 2023: 24; 2024: 17; 2025: 6; 2026: 2.

77 papers.

Sort:

By area:

  1. 2023Jailbreaking Black Box Large Language Models in Twenty Queriescited by 1,326
  2. 2022Red Teaming Language Models with Language Modelscited by 983
  3. 2022Red Teaming Language Models to Reduce Harms: Methods, Scaling Behaviors, and Lessons Learnedcited by 702
  4. 2023On Evaluating Adversarial Robustness of Large Vision-Language Modelscited by 281
  5. 2022WANLI: Worker and AI Collaboration for Natural Language Inference Dataset Creationcited by 200
  6. 2023Explore, Establish, Exploit: Red Teaming Language Models from Scratchcited by 111
  7. 2022State-of-the-art generalisation research in NLP: A taxonomy and reviewcited by 97
  8. 2025Findings of the BabyLM Challenge: Sample-Efficient Pretraining on Developmentally Plausible Corporacited by 86
  9. 2022Adversarial Training for High-Stakes Reliabilitycited by 60
  10. 2022Generating Data to Mitigate Spurious Correlations in Natural Language Inference Datasetscited by 54
  11. 2023On Robustness of Prompt-based Semantic Parsing with Large Pre-trained Language Model: An Empirical Study on Codexcited by 41
  12. 2021Turning Tables: Generating Examples from Semi-structured Tables for Endowing Language Models with Reasoning Skillscited by 39
38 more
  1. 2021Models in the Loop: Aiding Crowdworkers with Generative Annotation Assistantscited by 31
  2. 2023Query-Efficient Black-Box Red Teaming via Bayesian Optimizationcited by 31
  3. 2021Retrieval-guided Counterfactual Generation for QAcited by 28
  4. 2022Generative Language Models for Paragraph-Level Question Generationcited by 28
  5. 2024SimRAG: Self-Improving Retrieval-Augmented Generation for Adapting Large Language Models to Specialized Domainscited by 28
  6. 2024Protecting Your LLMs with Information Bottleneckcited by 26
  7. 2022QAmeleon: Multilingual QA with Only 5 Examplescited by 25
  8. 2022CORE: A Retrieve-then-Edit Framework for Counterfactual Data Generationcited by 23
  9. 2025SWE-Synth: Synthesizing Verifiable Bug-Fix Data to Enable Large Language Models in Resolving Real-World Bugscited by 22
  10. 2021Question Answering Infused Pre-training of General-Purpose Contextualized Representationscited by 19
  11. 2024MirrorCheck: Efficient Adversarial Defense for Vision-Language Modelscited by 19
  12. 2023Evolving Diverse Red-team Language Models in Multi-round Multi-agent Gamescited by 17
  13. 2022Dynatask: A Framework for Creating Dynamic AI Benchmark Taskscited by 16
  14. 2024Efficient LLM Jailbreak via Adaptive Dense-to-sparse Constrained Optimizationcited by 16
  15. 2023An Empirical Comparison of LM-based Question and Answer Generation Methodscited by 15
  16. 2025Cancer-Myth: Evaluating Large Language Models on Patient Questions with False Presuppositionscited by 15
  17. 2022Low-Resource Dense Retrieval for Open-Domain Question Answering: A Comprehensive Surveycited by 14
  18. 2024Improving Adversarial Data Collection by Supporting Annotators: Lessons from GAHD, a German Hate Speech Datasetcited by 13
  19. 2022Defending Against Disinformation Attacks in Open-Domain Question Answeringcited by 12
  20. 2023ASSERT: Automated Safety Scenario Red Teaming for Evaluating the Robustness of Large Language Modelscited by 12
  21. 2023A Practical Toolkit for Multilingual Question and Answer Generationcited by 10
  22. 2025Single-pass Detection of Jailbreaking Input in Large Language Modelscited by 10
  23. 2023Can LLMs Augment Low-Resource Reading Comprehension Datasets? Opportunities and Challengescited by 9
  24. 2023SCENE: Self-Labeled Counterfactuals for Extrapolating to Negative Examplescited by 8
  25. 2022Can In-context Learners Learn a Reasoning Concept from Demonstrations?cited by 7
  26. 2023Robust Training for Conversational Question Answering Models with Reinforced Reformulation Generationcited by 7
  27. 2024Is your benchmark truly adversarial? AdvScore: Evaluating Human-Grounded Adversarialnesscited by 7
  28. 2024An Empirical Comparison of Generative Approaches for Product Attribute-Value Identificationcited by 7
  29. 2023Resources and Few-shot Learners for In-context Learning in Slavic Languagescited by 6
  30. 2023Attributable and Scalable Opinion Summarizationcited by 6
  31. 2023PromptAttack: Probing Dialogue State Trackers with Adversarial Promptscited by 6
  32. 2022Event-Centric Question Answering via Contrastive Learning and Invertible Event Transformationcited by 5
  33. 2023Can Large Language Models Change User Preference Adversarially?cited by 5
  34. 2023Generative Data Augmentation using LLMs improves Distributional Robustness in Question Answeringcited by 5
  35. 2023Hallucination Augmented Recitations for Language Modelscited by 5
  36. 2024Benchmark Granularity and Model Robustness for Image-Text Retrievalcited by 5
  37. 2022A Survey on Measuring and Mitigating Reasoning Shortcuts in Machine Reading Comprehensioncited by 4
  38. 2022IDK-MRC: Unanswerable Questions for Indonesian Machine Reading Comprehensioncited by 4

50 most cited of 77 All 77 (JSON)