mlsanityData to 7 Oct 2026

An Empirical Survey of Data Augmentation for Limited Data Learning in NLP

arXiv 2106.07499 ↗First posted 14 Jun 2021cs.CL cs.AIcited by 101

Not on the map. Why?

Abstract (arXiv)

NLP has achieved great progress in the past decade through the use of neural models and large labeled datasets. The dependence on abundant data prevents NLP models from being applied to low-resource settings or novel tasks where significant time, money, or expertise is required to label massive amounts of textual data. Recently, data augmentation methods have been explored as a means of improving data efficiency in NLP. To date, there has been no systematic empirical overview of data augmentation for NLP in the limited labeled data setting, making it difficult to understand which methods work in which settings. In this paper, we provide an empirical survey of recent progress on data augmentation for NLP in the limited labeled data setting, summarizing the landscape of methods (including token-level augmentations, sentence-level augmentations, adversarial augmentations, and hidden-space augmentations) and carrying out experiments on 11 datasets covering topics/news classification, inference tasks, paraphrasing tasks, and single-sentence tasks. Based on the results, we draw several conclusions to help practitioners choose appropriate augmentations in different settings and discuss the current challenges and future directions for limited data learning in NLP.

Key facts. Cited by 101 papers in the map (data through 7 Oct 2026). Builds on 112 of its 170 references that are in the map. First posted on arXiv 14 Jun 2021.

Builds on

112 of 170 references.

References · 112

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2020Language Models are Few-Shot Learnerscited by 32,199
  3. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
  4. 2014Explaining and Harnessing Adversarial Examplescited by 7,287
  5. 2017mixup: Beyond Empirical Risk Minimizationcited by 5,155
  6. 2018GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understandingcited by 4,854
  7. 2014Sequence to Sequence Learning with Neural Networkscited by 2,974
  8. 2015Character-level Convolutional Networks for Text Classificationcited by 2,293
  9. 2018Deep contextualized word representationscited by 2,285
  10. 2020FixMatch: Simplifying Semi-Supervised Learning with Consistency and Confidencecited by 2,080
  11. 2017Improved Regularization of Convolutional Neural Networks with Cutoutcited by 1,623
  12. 2019MixMatch: A Holistic Approach to Semi-Supervised Learningcited by 1,390
38 more
  1. 2017Random Erasing Data Augmentationcited by 1,235
  2. 2019Cross-lingual Language Model Pretrainingcited by 1,088
  3. 2019Unsupervised Data Augmentation for Consistency Trainingcited by 1,057
  4. 2016Temporal Ensembling for Semi-Supervised Learningcited by 1,054
  5. 2020Exploiting Cloze Questions for Few Shot Text Classification and Natural Language Inferencecited by 934
  6. 2017Virtual Adversarial Training: A Regularization Method for Supervised and Semi-Supervised Learningcited by 918
  7. 2019Right for the Wrong Reasons: Diagnosing Syntactic Heuristics in Natural Language Inferencecited by 864
  8. 2019EDA: Easy Data Augmentation Techniques for Boosting Performance on Text Classification Taskscited by 800
  9. 2015Improving Neural Machine Translation Models with Monolingual Datacited by 784
  10. 2020It's Not Just Size That Matters: Small Language Models Are Also Few-Shot Learnerscited by 668
  11. 2015Generating Sentences from a Continuous Spacecited by 607
  12. 2017Adversarial Examples for Evaluating Reading Comprehension Systemscited by 568
  13. 2014Semi-Supervised Learning with Deep Generative Modelscited by 494
  14. 2016Google's Multilingual Neural Machine Translation System: Enabling Zero-Shot Translationcited by 467
  15. 2019Generalizing from a Few Examples: A Survey on Few-Shot Learningcited by 462
  16. 2018Generating Natural Language Adversarial Examplescited by 422
  17. 2021A Survey of Data Augmentation Approaches for NLPcited by 408
  18. 2020BAE: BERT-based Adversarial Examples for Text Classificationcited by 316
  19. 2017Synthetic and Natural Noise Both Break Neural Machine Translationcited by 304
  20. 2018Adversarial Example Generation with Syntactically Controlled Paraphrase Networkscited by 282
  21. 2016Adversarial Training Methods for Semi-Supervised Text Classificationcited by 280
  22. 2017Unsupervised Machine Translation Using Monolingual Corpora Onlycited by 272
  23. 2017Toward Controlled Generation of Textcited by 271
  24. 2019FreeLB: Enhanced Adversarial Training for Natural Language Understandingcited by 261
  25. 2018FewRel: A Large-Scale Supervised Few-Shot Relation Classification Dataset with State-of-the-Art Evaluationcited by 254
  26. 2019SMART: Robust and Efficient Fine-Tuning for Pre-trained Natural Language Models through Principled Regularized Optimizationcited by 253
  27. 2014Learning with Pseudo-Ensemblescited by 241
  28. 2018Contextual Augmentation: Data Augmentation by Words with Paradigmatic Relationscited by 201
  29. 2020MixText: Linguistically-Informed Interpolation of Hidden Space for Semi-Supervised Text Classificationcited by 186
  30. 2017Generating Natural Adversarial Examplescited by 182
  31. 2019Not Enough Data? Deep Learning to the Rescue!cited by 174
  32. 2020A Survey on Recent Approaches for Natural Language Processing in Low-Resource Scenarioscited by 170
  33. 2020Data Augmentation using Pre-trained Transformer Modelscited by 168
  34. 2018QANet: Combining Local Convolution with Global Self-Attention for Reading Comprehensioncited by 164
  35. 2019Good-Enough Compositional Data Augmentationcited by 158
  36. 2019Revisiting Self-Training for Neural Sequence Generationcited by 154
  37. 2020Compositional Generalization and Natural Language Variation: Can a Semantic Parsing Approach Handle Both?cited by 147
  38. 2020Deep Learning Based Text Classification: A Comprehensive Reviewcited by 140

50 most cited of 112 All 112 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 11; 2022: 18; 2023: 19; 2024: 31; 2025: 10; 2026: 12.

101 papers.

Sort:

By area:

  1. 2021Recent Advances in Natural Language Processing via Large Pre-Trained Language Models: A Surveycited by 457
  2. 2022Is GPT-3 a Good Data Annotator?cited by 208
  3. 2021NL-Augmenter: A Framework for Task-Sensitive Natural Language Augmentationcited by 72
  4. 2024Y Social: an LLM-powered Social Media Digital Twincited by 43
  5. 2021Attacking Open-domain Question Answering by Injecting Misinformationcited by 41
  6. 2024What is the Role of Small Models in the LLM Era: A Surveycited by 41
  7. 2022DoCoGen: Domain Counterfactual Generation for Low Resource Domain Adaptationcited by 36
  8. 2023Large Language Models as Annotators: Enhancing Generalization of NLP Models at Minimal Costcited by 33
  9. 2022Weakly Supervised Data Augmentation Through Prompting for Dialogue Understandingcited by 32
  10. 2022Continual Sequence Generation with Adaptive Compositional Modulescited by 31
  11. 2024Data Augmentation using Large Language Models: Data Perspectives, Learning Paradigms and Challengescited by 30
  12. 2024Prompt Perturbation Consistency Learning for Robust Language Modelscited by 26
38 more
  1. 2024On Behalf of the Stakeholders: Trends in NLP Model Interpretability in the Era of LLMscited by 24
  2. 2024A Comparative Study on Enhancing Prediction in Social Network Advertisement through Data Augmentationcited by 20
  3. 2021Parametric Scattering Networkscited by 15
  4. 2021Text AutoAugment: Learning Compositional Augmentation Policy for Text Classificationcited by 15
  5. 2022TreeMix: Compositional Constituency-based Data Augmentation for Natural Language Understandingcited by 15
  6. 2024Cheetah: Natural Language Generation for 517 African Languagescited by 15
  7. 2024Tabular Data Augmentation for Machine Learning: Progress and Prospects of Embracing Generative AIcited by 15
  8. 2025Charting the Landscape of African NLP: Mapping Progress and Shaping the Road Aheadcited by 15
  9. 2022HyperMixer: An MLP-based Low Cost Alternative to Transformerscited by 13
  10. 2021To Augment or Not to Augment? A Comparative Study on Text Augmentation Techniques for Low-Resource NLPcited by 12
  11. 2023Ada-Instruct: Adapting Instruction Generators for Complex Reasoningcited by 11
  12. 2021DaCy: A Unified Framework for Danish NLPcited by 10
  13. 2021ALP: Data Augmentation using Lexicalized PCFGs for Few-Shot Text Classificationcited by 10
  14. 2022Information Extraction in Low-Resource Scenarios: Survey and Perspectivecited by 10
  15. 2024CoDa: Constrained Generation based Data Augmentation for Low-Resource NLPcited by 9
  16. 2025Where Are We? Evaluating LLM Performance on African Languagescited by 9
  17. 2022Improving Meta-learning for Low-resource Text Classification and Generation via Memory Imitationcited by 8
  18. 2022Explanation Graph Generation via Pre-trained Language Models: An Empirical Study with Contrastive Learningcited by 8
  19. 2024Improving Black-box Robustness with In-Context Rewritingcited by 8
  20. 2024ABEX: Data Augmentation for Low-Resource NLU via Expanding Abstract Descriptionscited by 8
  21. 2024FairDgcl: Fairness-aware Recommendation with Dynamic Graph Contrastive Learningcited by 8
  22. 2021Data Augmentation for Mental Health Classification on Social Mediacited by 7
  23. 2022PromptDA: Label-guided Data Augmentation for Prompt-based Few-shot Learnerscited by 7
  24. 2022Data Augmentation for Dementia Detection in Spoken Languagecited by 7
  25. 2023Fabricator: An Open Source Toolkit for Generating Labeled Training Data with Teacher LLMscited by 7
  26. 2024A Comprehensive Study on NLP Data Augmentation for Hate Speech Detection: Legacy Methods, BERT, and LLMscited by 7
  27. 2024A Survey on Data Quality Dimensions and Tools for Machine Learningcited by 7
  28. 2024Synthio: Augmenting Small-Scale Audio Classification Datasets with Synthetic Datacited by 7
  29. 2022Counterfactual Data Augmentation via Perspective Transition for Open-Domain Dialoguescited by 6
  30. 2024Enhancing In-Context Learning via Implicit Demonstration Augmentationcited by 6
  31. 2024Recent Advancements and Challenges of Turkic Central Asian Language Processingcited by 6
  32. 2022Disambiguation of morpho-syntactic features of African American English -- the case of habitual becited by 5
  33. 2022Selective Text Augmentation with Word Roles for Low-Resource Text Classificationcited by 5
  34. 2023BioAug: Conditional Generation based Data Augmentation for Low-Resource Biomedical NERcited by 5
  35. 2024Domain-adaptative Continual Learning for Low-resource Tasks: Evaluation on Nepalicited by 5
  36. 2023A comprehensive review of automatic text summarization techniques: method, data, evaluation and codingcited by 4
  37. 2023MixPro: Simple yet Effective Data Augmentation for Prompt-based Learningcited by 4
  38. 2024Monolingual and Multilingual Misinformation Detection for Low-Resource Languages: A Comprehensive Surveycited by 4

50 most cited of 101 All 101 (JSON)