mlsanityData to 7 Oct 2026

A Survey of Data Augmentation Approaches for NLP

arXiv 2105.03075 ↗First posted 7 May 2021cs.CL cs.AI cs.LGcited by 408

Not on the map. Why?

Abstract (arXiv)

Data augmentation has recently seen increased interest in NLP due to more work in low-resource domains, new tasks, and the popularity of large-scale neural networks that require large amounts of training data. Despite this recent upsurge, this area is still relatively underexplored, perhaps due to the challenges posed by the discrete nature of language data. In this paper, we present a comprehensive and unifying survey of data augmentation for NLP by summarizing the literature in a structured manner. We first introduce and motivate data augmentation for NLP, and then discuss major methodologically representative approaches. Next, we highlight techniques that are used for popular NLP applications and tasks. We conclude by outlining current challenges and directions for future research. Overall, our paper aims to clarify the landscape of existing literature in data augmentation for NLP and motivate additional work in this area. We also present a GitHub repository with a paper list that will be continuously updated at https://github.com/styfeng/DataAug4NLP

Key facts. Cited by 408 papers in the map (data through 7 Oct 2026). Builds on 119 of its 193 references that are in the map. First posted on arXiv 7 May 2021.

Builds on

119 of 193 references.

References · 119

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 11,979
  3. 2019BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehensioncited by 5,874
  4. 2017mixup: Beyond Empirical Risk Minimizationcited by 5,155
  5. 2019CutMix: Regularization Strategy to Train Strong Classifiers with Localizable Featurescited by 2,875
  6. 2015Character-level Convolutional Networks for Text Classificationcited by 2,293
  7. 2019RandAugment: Practical automated data augmentation with a reduced search spacecited by 2,261
  8. 2017Improved Regularization of Convolutional Neural Networks with Cutoutcited by 1,623
  9. 2019Unsupervised Data Augmentation for Consistency Trainingcited by 1,057
  10. 2015Teaching Machines to Read and Comprehendcited by 1,039
  11. 2019EDA: Easy Data Augmentation Techniques for Boosting Performance on Text Classification Taskscited by 800
  12. 2015Improving Neural Machine Translation Models with Monolingual Datacited by 784
38 more
  1. 2018AutoAugment: Learning Augmentation Policies from Datacited by 728
  2. 2018Gender Bias in Coreference Resolution: Evaluation and Debiasing Methodscited by 657
  3. 2020Simple Copy-Paste is a Strong Data Augmentation Method for Instance Segmentationcited by 595
  4. 2020TAPAS: Weakly Supervised Table Parsing via Pre-trainingcited by 432
  5. 2019Learning the Difference that Makes a Difference with Counterfactually-Augmented Datacited by 428
  6. 2019PAWS: Paraphrase Adversaries from Word Scramblingcited by 395
  7. 2018Adversarial Example Generation with Syntactically Controlled Paraphrase Networkscited by 282
  8. 2017Challenges in Data-to-Document Generationcited by 226
  9. 2018Colorless green recurrent networks dream hierarchicallycited by 210
  10. 2018Gender Bias in Neural Natural Language Processingcited by 210
  11. 2018Contextual Augmentation: Data Augmentation by Words with Paradigmatic Relationscited by 201
  12. 2019Counterfactual Data Augmentation for Mitigating Gender Stereotypes in Languages with Rich Morphologycited by 200
  13. 2019Not Enough Data? Deep Learning to the Rescue!cited by 174
  14. 2020GraPPa: Grammar-Augmented Pre-Training for Table Semantic Parsingcited by 171
  15. 2019Certified Robustness to Adversarial Word Substitutionscited by 164
  16. 2019Good-Enough Compositional Data Augmentationcited by 158
  17. 2017Data Augmentation for Low-Resource Neural Machine Translationcited by 139
  18. 2020Augmented SBERT: Data Augmentation Method for Improving Bi-Encoders for Pairwise Sentence Scoring Taskscited by 136
  19. 2018Breaking NLI Systems with Sentences that Require Simple Lexical Inferencescited by 134
  20. 2020An Empirical Survey of Data Augmentation for Time Series Classification with Neural Networkscited by 128
  21. 2016Data Recombination for Neural Semantic Parsingcited by 111
  22. 2018A Kernel Theory of Modern Data Augmentationcited by 106
  23. 2019It's All in the Name: Mitigating Gender Bias with Name-Based Counterfactual Data Substitutioncited by 102
  24. 2019Task-Oriented Dialog Systems that Consider Multiple Appropriate Responses under the Same Contextcited by 99
  25. 2020SSMBA: Self-Supervised Manifold Based Data Augmentation for Improving Out-of-Domain Robustnesscited by 94
  26. 2017Learning to Compose Domain-Specific Transformations for Data Augmentationcited by 91
  27. 2018Delta-encoder: an effective sample synthesis method for few-shot object recognitioncited by 89
  28. 2019Editing-Based SQL Query Generation for Cross-Domain Context-Dependent Questionscited by 80
  29. 2020CoSDA-ML: Multi-Lingual Code-Switching Data Augmentation for Zero-Shot Cross-Lingual NLPcited by 79
  30. 2020Logic-Guided Data Augmentation and Regularization for Consistent Question Answeringcited by 77
  31. 2020Grounded Adaptation for Zero-shot Executable Semantic Parsingcited by 76
  32. 2020An Analysis of Simple Data Augmentation for Named Entity Recognitioncited by 75
  33. 2018SwitchOut: an Efficient Data Augmentation Algorithm for Neural Machine Translationcited by 72
  34. 2020Sequence-Level Mixed Sample Data Augmentationcited by 71
  35. 2018Sequence-to-Sequence Data Augmentation for Dialogue Language Understandingcited by 67
  36. 2020AdvAug: Robust Adversarial Augmentation for Neural Machine Translationcited by 66
  37. 2020Zero-Shot Transfer Learning with Synthesized Data for Multi-Domain Dialogue State Trackingcited by 65
  38. 2020Explaining The Efficacy of Counterfactually Augmented Datacited by 64

50 most cited of 119 All 119 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 34; 2022: 115; 2023: 107; 2024: 94; 2025: 39; 2026: 19.

408 papers.

Sort:

By area:

  1. 2022Self-Instruct: Aligning Language Models with Self-Generated Instructionscited by 2,674
  2. 2022Discovering Language Model Behaviors with Model-Written Evaluationscited by 805
  3. 2023RLAIF vs. RLHF: Scaling Reinforcement Learning from Human Feedback with AI Feedbackcited by 615
  4. 2021Recent Advances in Natural Language Processing via Large Pre-Trained Language Models: A Surveycited by 457
  5. 2023A Survey on Large Language Model (LLM) Security and Privacy: The Good, the Bad, and the Uglycited by 380
  6. 2024A Survey on Knowledge Distillation of Large Language Modelscited by 276
  7. 2022Is GPT-3 a Good Data Annotator?cited by 208
  8. 2022Learning Video Representations from Large Language Modelscited by 204
  9. 2023REFINER: Reasoning Feedback on Intermediate Representationscited by 203
  10. 2023Data-centric Artificial Intelligence: A Surveycited by 202
  11. 2023AugGPT: Leveraging ChatGPT for Text Data Augmentationcited by 128
  12. 2021A Survey on Data Augmentation for Text Classificationcited by 111
38 more
  1. 2021Measure and Improve Robustness in NLP Models: A Surveycited by 107
  2. 2021An Empirical Survey of Data Augmentation for Limited Data Learning in NLPcited by 101
  3. 2023Waffling around for Performance: Visual Classification with Random Words and Broad Conceptscited by 100
  4. 2021Data Augmentation Approaches in Natural Language Processing: A Surveycited by 82
  5. 2023A Survey of GPT-3 Family Large Language Models Including ChatGPT and GPT-4cited by 81
  6. 2023ReLLa: Retrieval-enhanced Large Language Models for Lifelong Sequential Behavior Comprehension in Recommendationcited by 76
  7. 2024A Survey of Multimodal Large Language Model from A Data-centric Perspectivecited by 73
  8. 2021NL-Augmenter: A Framework for Task-Sensitive Natural Language Augmentationcited by 72
  9. 2022Formal Algorithms for Transformerscited by 70
  10. 2024LLM2LLM: Boosting LLMs with Novel Iterative Data Enhancementcited by 69
  11. 2021DialFact: A Benchmark for Fact-Checking in Dialoguecited by 62
  12. 2022Graph Data Augmentation for Graph Machine Learning: A Surveycited by 61
  13. 2022PromDA: Prompt-based Data Augmentation for Low-Resource NLU Taskscited by 51
  14. 2022Spectral Feature Augmentation for Graph Contrastive Learning and Beyondcited by 49
  15. 2023MuggleMath: Assessing the Impact of Query and Response Augmentation on Math Reasoningcited by 45
  16. 2023Data-centric AI: Perspectives and Challengescited by 43
  17. 2021Generate, Annotate, and Learn: NLP with Synthetic Textcited by 42
  18. 2022Data Augmentation techniques in time series domain: A survey and taxonomycited by 41
  19. 2024A Survey on Data Synthesis and Augmentation for Large Language Modelscited by 40
  20. 2021SSMix: Saliency-Based Span Mixup for Text Classificationcited by 39
  21. 2021Turning Tables: Generating Examples from Semi-structured Tables for Endowing Language Models with Reasoning Skillscited by 39
  22. 2022COSTA: Covariance-Preserving Feature Augmentation for Graph Contrastive Learningcited by 38
  23. 2022Compute-Efficient Deep Learning: Algorithmic Trends and Opportunitiescited by 38
  24. 2024A Comprehensive Survey on Data Augmentationcited by 38
  25. 2023Generating Faithful Synthetic Data with Large Language Models: A Case Study in Computational Social Sciencecited by 37
  26. 2022DoCoGen: Domain Counterfactual Generation for Low Resource Domain Adaptationcited by 36
  27. 2022AugLy: Data Augmentations for Robustnesscited by 34
  28. 2022On the Evaluation Metrics for Paraphrase Generationcited by 32
  29. 2022Data augmentation for learning predictive models on EEG: a systematic comparisoncited by 32
  30. 2023Are Large Language Models Really Robust to Word-Level Perturbations?cited by 32
  31. 2024A Survey on Data Augmentation in Large Model Eracited by 31
  32. 2022A Survey of Mix-based Data Augmentation: Taxonomy, Methods, Applications, and Explainabilitycited by 30
  33. 2024Data Augmentation using Large Language Models: Data Perspectives, Learning Paradigms and Challengescited by 30
  34. 2024Unleashing the Power of Data Tsunami: A Comprehensive Survey on Data Assessment and Selection for Instruction Tuning of Language Modelscited by 29
  35. 2021Imbalanced Graph Classification via Graph-of-Graph Neural Networkscited by 27
  36. 2023On Efficient Training of Large-Scale Deep Learning Models: A Literature Reviewcited by 26
  37. 2024Prompt Perturbation Consistency Learning for Robust Language Modelscited by 26
  38. 2023CHBias: Bias Evaluation and Mitigation of Chinese Conversational Language Modelscited by 25

50 most cited of 408 All 408 (JSON)