mlsanityData to 14 Oct 2026

Training Question Answering Models From Synthetic Data

arXiv 2002.09599 ↗First posted 22 Feb 2020cs.CL cs.AIcited by 95

Not on the map. Why?

Abstract (arXiv)

Question and answer generation is a data augmentation method that aims to improve question answering (QA) models given the limited amount of human labeled data. However, a considerable gap remains between synthetic and human-generated question-answer pairs. This work aims to narrow this gap by taking advantage of large language models and explores several factors such as model size, quality of pretrained models, scale of data synthesized, and algorithmic choices. On the SQuAD1.1 question answering task, we achieve higher accuracy using solely synthetic questions and answers than when using the SQuAD1.1 training set questions alone. Removing access to real Wikipedia data, we synthesize questions and answers from a synthetic corpus generated by an 8.3 billion parameter GPT-2 model. With no access to human supervision and only access to other models, we are able to train state of the art question answering networks on entirely model-generated data that achieve 88.4 Exact Match (EM) and 93.9 F1 score on the SQuAD1.1 dev set. We further apply our methodology to SQuAD2.0 and show a 2.8 absolute gain on EM score compared to prior work using synthetic data.

Key facts. Cited by 95 papers in the map (data through 14 Oct 2026). First posted on arXiv 22 Feb 2020.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2021: 16; 2022: 24; 2023: 18; 2024: 26; 2025: 6; 2026: 5.

95 papers.

Sort:

By area:

  1. 2022BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and Generationcited by 4,322
  2. 2025Towards Large Reasoning Models: A Survey of Reinforced Reasoning with Large Language Modelscited by 293
  3. 2022ZeroGen: Efficient Zero-shot Learning via Dataset Generationcited by 211
  4. 2021Improving Question Answering Model Robustness with Synthetic Adversarial Data Generationcited by 79
  5. 2022ProGen: Progressive Zero-shot Dataset Generation via In-context Feedbackcited by 67
  6. 2022All You May Need for VQA are Image Captionscited by 65
  7. 2023Leveraging Large Language Models in Conversational Recommender Systemscited by 63
  8. 2021Few-Shot Question Answering by Pretraining Span Selectioncited by 62
  9. 2024When Search Engine Services meet Large Language Models: Visions and Challengescited by 59
  10. 2022Self-Guided Noise-Free Data Generation for Efficient Zero-Shot Learningcited by 50
  11. 2022Automatic Generation of Socratic Subquestions for Teaching Math Word Problemscited by 43
  12. 2021Turning Tables: Generating Examples from Semi-structured Tables for Endowing Language Models with Reasoning Skillscited by 39
38 more
  1. 2021STraTA: Self-Training with Task Augmentation for Better Few-shot Learningcited by 33
  2. 2021Models in the Loop: Aiding Crowdworkers with Generative Annotation Assistantscited by 31
  3. 2021ArchivalQA: A Large-scale Benchmark Dataset for Open Domain Question Answering over Historical News Collectionscited by 30
  4. 2023Let's Synthesize Step by Step: Iterative Dataset Synthesis with Large Language Models by Extrapolating Errors from Small Modelscited by 30
  5. 2022Generative Language Models for Paragraph-Level Question Generationcited by 28
  6. 2021FewshotQA: A simple framework for few-shot learning of question answering tasks using pre-trained text-to-text modelscited by 26
  7. 2021Improving Unsupervised Question Answering via Summarization-Informed Question Generationcited by 21
  8. 2021Contrastive Domain Adaptation for Question Answering using Limited Text Corporacited by 20
  9. 2021Question Answering Infused Pre-training of General-Purpose Contextualized Representationscited by 19
  10. 2023LIQUID: A Framework for List Question Answering Dataset Generationcited by 19
  11. 2021SynthBio: A Case Study in Human-AI Collaborative Curation of Text Datasetscited by 18
  12. 2022Generating Information-Seeking Conversations from Unlabeled Documentscited by 17
  13. 2022RQUGE: Reference-Free Metric for Evaluating Question Generation by Answering the Questioncited by 17
  14. 2022Learning to Ask Like a Physiciancited by 16
  15. 2023An Empirical Comparison of LM-based Question and Answer Generation Methodscited by 15
  16. 2024A synthetic data approach for domain generalization of NLI modelscited by 15
  17. 2023UniGen: A Unified Generative Framework for Retrieval and Question Answering with Large Language Modelscited by 14
  18. 2024textitTrans-LoRA: towards data-free Transferable Parameter Efficient Finetuningcited by 13
  19. 2022Synthetic Question Value Estimation for Domain Adaptation of Question Answeringcited by 12
  20. 2022Leveraging QA Datasets to Improve Generative Data Augmentationcited by 12
  21. 2023MinPrompt: Graph-based Minimal Prompt Data Augmentation for Few-shot Question Answeringcited by 12
  22. 2024Can LLMs Replace Economic Choice Prediction Labs? The Case of Language-based Persuasion Gamescited by 12
  23. 2024LegalLens: Leveraging LLMs for Legal Violation Identification in Unstructured Textcited by 12
  24. 2023A Practical Toolkit for Multilingual Question and Answer Generationcited by 10
  25. 2022ED2LM: Encoder-Decoder to Language Model for Faster Document Re-ranking Inferencecited by 9
  26. 2022QAScore -- An Unsupervised Unreferenced Metric for the Question Generation Evaluationcited by 9
  27. 2024Prompting-based Synthetic Data Generation for Few-Shot Question Answeringcited by 9
  28. 2024A Survey of Data Synthesis Approachescited by 9
  29. 2024DiffLM: Controllable Synthetic Data Generation via Diffusion Language Modelscited by 9
  30. 2023Does Synthetic Data Make Large Language Models More Efficient?cited by 8
  31. 2024Synthetic Multimodal Question Generationcited by 8
  32. 2024Advancing Post-OCR Correction: A Comparative Study of Synthetic Datacited by 8
  33. 2024LlamaDuo: LLMOps Pipeline for Seamless Migration from Service LLMs to Small-Scale Local LLMscited by 8
  34. 2024Synthetic Data Generation with Large Language Models for Personalized Community Question Answeringcited by 8
  35. 2021Topic Transferable Table Question Answeringcited by 7
  36. 2021Relation-Guided Pre-Training for Open-Domain Question Answeringcited by 7
  37. 2023Robust Training for Conversational Question Answering Models with Reinforced Reformulation Generationcited by 7
  38. 2024Forewarned is Forearmed: Leveraging LLMs for Data Synthesis through Failure-Inducing Explorationcited by 7

50 most cited of 95 All 95 (JSON)