mlsanityData to 14 Oct 2026

Data Augmentation Approaches in Natural Language Processing: A Survey

arXiv 2110.01852 ↗First posted 5 Oct 2021cs.CL cs.AI cs.LGcited by 82

Not on the map. Why?

Abstract (arXiv)

As an effective strategy, data augmentation (DA) alleviates data scarcity scenarios where deep learning techniques may fail. It is widely applied in computer vision then introduced to natural language processing and achieves improvements in many tasks. One of the main focuses of the DA methods is to improve the diversity of training data, thereby helping the model to better generalize to unseen testing data. In this survey, we frame DA methods into three categories based on the diversity of augmented data, including paraphrasing, noising, and sampling. Our paper sets out to analyze DA methods in detail according to the above categories. Further, we also introduce their applications in NLP tasks as well as the challenges. Some helpful resources are provided in the appendix.

Key facts. Cited by 82 papers in the map (data through 14 Oct 2026). Builds on 103 of its 156 references that are in the map. First posted on arXiv 5 Oct 2021.

Builds on

103 of 156 references.

References · 103

  1. 2020Language Models are Few-Shot Learnerscited by 32,419
  2. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,599
  3. 2017mixup: Beyond Empirical Risk Minimizationcited by 5,183
  4. 2019DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lightercited by 4,307
  5. 2013Distributed Representations of Words and Phrases and their Compositionalitycited by 3,474
  6. 2015Character-level Convolutional Networks for Text Classificationcited by 2,330
  7. 2019TinyBERT: Distilling BERT for Natural Language Understandingcited by 1,167
  8. 2019Unsupervised Data Augmentation for Consistency Trainingcited by 1,060
  9. 2019EDA: Easy Data Augmentation Techniques for Boosting Performance on Text Classification Taskscited by 807
  10. 2019Generalization through Memorization: Nearest Neighbor Language Modelscited by 800
  11. 2015Improving Neural Machine Translation Models with Monolingual Datacited by 788
  12. 2021A Survey of Data Augmentation Approaches for NLPcited by 409
38 more
  1. 2021Internet-Augmented Dialogue Generationcited by 217
  2. 2018Contextual Augmentation: Data Augmentation by Words with Paradigmatic Relationscited by 201
  3. 2019Counterfactual Data Augmentation for Mitigating Gender Stereotypes in Languages with Rich Morphologycited by 200
  4. 2019Not Enough Data? Deep Learning to the Rescue!cited by 175
  5. 2020Data Augmentation using Pre-trained Transformer Modelscited by 168
  6. 2018QANet: Combining Local Convolution with Global Self-Attention for Reading Comprehensioncited by 164
  7. 2019Good-Enough Compositional Data Augmentationcited by 158
  8. 2017Data Augmentation for Low-Resource Neural Machine Translationcited by 139
  9. 2020Augmented SBERT: Data Augmentation Method for Improving Bi-Encoders for Pairwise Sentence Scoring Taskscited by 138
  10. 2021End-to-End Training of Multi-Document Reader and Retriever for Open-Domain Question Answeringcited by 133
  11. 2019Augmenting Data with Mixup for Sentence Classification: An Empirical Studycited by 125
  12. 2021A Survey on Data Augmentation for Text Classificationcited by 111
  13. 2018Conditional BERT Contextual Augmentationcited by 107
  14. 2018A Kernel Theory of Modern Data Augmentationcited by 106
  15. 2019Robust Neural Machine Translation with Doubly Adversarial Inputscited by 106
  16. 2020SSMBA: Self-Supervised Manifold Based Data Augmentation for Improving Out-of-Domain Robustnesscited by 96
  17. 2020Self-training Improves Pre-training for Natural Language Understandingcited by 96
  18. 2017Regularization for Deep Learning: A Taxonomycited by 91
  19. 2020Syntactic Data Augmentation Increases Robustness to Inference Heuristicscited by 86
  20. 2021Adaptive Semiparametric Language Modelscited by 83
  21. 2020CoSDA-ML: Multi-Lingual Code-Switching Data Augmentation for Zero-Shot Cross-Lingual NLPcited by 79
  22. 2017Data Noising as Smoothing in Neural Network Language Modelscited by 78
  23. 2020Logic-Guided Data Augmentation and Regularization for Consistent Question Answeringcited by 78
  24. 2020An Analysis of Simple Data Augmentation for Named Entity Recognitioncited by 76
  25. 2018SwitchOut: an Efficient Data Augmentation Algorithm for Neural Machine Translationcited by 72
  26. 2018Sequence-to-Sequence Data Augmentation for Dialogue Language Understandingcited by 68
  27. 2020AdvAug: Robust Adversarial Augmentation for Neural Machine Translationcited by 67
  28. 2019Generalized Data Augmentation for Low-Resource Translationcited by 53
  29. 2020Improving Zero and Few-Shot Abstractive Summarization with Intermediate Fine-tuning and Data Augmentationcited by 53
  30. 2019Soft Contextual Data Augmentation for Neural Machine Translationcited by 49
  31. 2020On the Generalization Effects of Linear Transformations in Data Augmentationcited by 47
  32. 2020Data Boost: Text Data Augmentation Through Reinforcement Learning Guided Conditional Generationcited by 47
  33. 2019Data Augmentation via Dependency Tree Morphing for Low-Resource Languagescited by 45
  34. 2019Learning Data Manipulation for Augmentation and Weightingcited by 45
  35. 2020TextAttack: A Framework for Adversarial Attacks, Data Augmentation, and Adversarial Training in NLPcited by 45
  36. 2020How Effective is Task-Agnostic Data Augmentation for Pretrained Transformers?cited by 41
  37. 2020CoDA: Contrast-enhanced and Diversity-promoting Data Augmentation for Natural Language Understandingcited by 41
  38. 2020Synthetic Data Augmentation for Zero-Shot Cross-Lingual Question Answeringcited by 41

50 most cited of 103 All 103 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2022: 17; 2023: 30; 2024: 19; 2025: 12; 2026: 4.

82 papers.

Sort:

By area:

  1. 2022Survey of Hallucination in Natural Language Generationcited by 2,075
  2. 2023The Curious Decline of Linguistic Diversity: Training Language Models on Synthetic Textcited by 100
  3. 2023A Survey of GPT-3 Family Large Language Models Including ChatGPT and GPT-4cited by 81
  4. 2023ReLLa: Retrieval-enhanced Large Language Models for Lifelong Sequential Behavior Comprehension in Recommendationcited by 77
  5. 2024A Survey of Mathematical Reasoning in the Era of Multimodal Large Language Model: Benchmark, Method & Challengescited by 55
  6. 2022A Survey of Mix-based Data Augmentation: Taxonomy, Methods, Applications, and Explainabilitycited by 31
  7. 2023Let's Synthesize Step by Step: Iterative Dataset Synthesis with Large Language Models by Extrapolating Errors from Small Modelscited by 30
  8. 2024InternalInspector I2: Robust Confidence Estimation in LLMs through Internal Statescited by 30
  9. 2024Unleashing the Power of Data Tsunami: A Comprehensive Survey on Data Assessment and Selection for Instruction Tuning of Language Modelscited by 29
  10. 2023A review of ensemble learning and data augmentation models for class imbalanced problems: combination, implementation and evaluationcited by 25
  11. 2023Noise-Robust Fine-Tuning of Pretrained Language Models via External Guidancecited by 24
  12. 2023A Ship of Theseus: Curious Cases of Paraphrasing in LLM-Generated Textscited by 23
38 more
  1. 2025Text Data Augmentation for Large Language Models: A Comprehensive Survey of Methods, Challenges, and Opportunitiescited by 22
  2. 2023Using GPT-4 to Augment Unbalanced Data for Automatic Scoringcited by 20
  3. 2023Semantic-Guided Generative Image Augmentation Method with Diffusion Models for Image Classificationcited by 16
  4. 2024Retrieval-Augmented Data Augmentation for Low-Resource Domain Taskscited by 12
  5. 2024Data Generation Using Large Language Models for Text Classification: An Empirical Case Studycited by 11
  6. 2023GDA: Generative Data Augmentation Techniques for Relation Extraction Taskscited by 10
  7. 2024Face4RAG: Factual Consistency Evaluation for Retrieval Augmented Generation in Chinesecited by 10
  8. 2023Enhancing Abstractiveness of Summarization Models through Calibrated Distillationcited by 8
  9. 2024Advancing Post-OCR Correction: A Comparative Study of Synthetic Datacited by 8
  10. 2024Rephrasing natural text data with different languages and quality levels for Large Language Model pre-trainingcited by 8
  11. 2022PromptDA: Label-guided Data Augmentation for Prompt-based Few-shot Learnerscited by 7
  12. 2022An Efficient Framework for Few-shot Skeleton-based Temporal Action Segmentationcited by 7
  13. 2022FactMix: Using a Few Labeled In-domain Examples to Generalize to Cross-domain Named Entity Recognitioncited by 7
  14. 2022A Survey on Table-and-Text HybridQA: Concepts, Methods, Challenges and Future Directionscited by 7
  15. 2024A Survey on Data Quality Dimensions and Tools for Machine Learningcited by 7
  16. 2022Entity-to-Text based Data Augmentation for various Named Entity Recognition Taskscited by 6
  17. 2023Data Optimization in Deep Learning: A Surveycited by 6
  18. 2023Target-Side Augmentation for Document-Level Machine Translationcited by 5
  19. 2025Enhancing Arabic Automated Essay Scoring with Synthetic Data and Error Injectioncited by 5
  20. 2025Explainable AI: XAI-Guided Context-Aware Data Augmentationcited by 5
  21. 2022AdMix: A Mixed Sample Data Augmentation Method for Neural Machine Translationcited by 4
  22. 2022Joint Generator-Ranker Learning for Natural Language Generationcited by 4
  23. 2023MixPro: Simple yet Effective Data Augmentation for Prompt-based Learningcited by 4
  24. 2023Conversational Process Modeling: Can Generative AI Empower Domain Experts in Creating and Redesigning Process Models?cited by 4
  25. 2024Research and application of artificial intelligence based webshell detection model: A literature reviewcited by 4
  26. 2026MeMo: Memory as a Modelcited by 4
  27. 2026Data Augmentation: A Fourier Analysis Perspectivecited by 4
  28. 2022Data Augmentation using Transformers and Similarity Measures for Improving Arabic Text Classificationcited by 3
  29. 2023A Communication Theory Perspective on Prompting Engineering Methods for Large Language Modelscited by 3
  30. 2023A Multi-solution Study on GDPR AI-enabled Completeness Checking of DPAscited by 3
  31. 2024Modeling Orthographic Variation Improves NLP Performance for Nigerian Pidgincited by 3
  32. 2022Knowledge-Grounded Conversational Data Augmentation with Generative Conversational Networkscited by 2
  33. 2022Check-worthy Claim Detection across Topics for Automated Fact-checkingcited by 2
  34. 2023Data Augmentation for Neural NLPcited by 2
  35. 2023HULAT at SemEval-2023 Task 10: Data augmentation for pre-trained transformers applied to the detection of sexism in social mediacited by 2
  36. 2024Channel Reflection: Knowledge-Driven Data Augmentation for EEG-Based Brain-Computer Interfacescited by 2
  37. 2025MASTER: Enhancing Large Language Model via Multi-Agent Simulated Teachingcited by 2
  38. 2022Fast and Accurate FSA System Using ELBERT: An Efficient and Lightweight BERTcited by 1

50 most cited of 82 All 82 (JSON)