mlsanityData to 14 Oct 2026

Data Augmentation using Large Language Models: Data Perspectives, Learning Paradigms and Challenges

arXiv 2403.02990 ↗First posted 5 Mar 2024cs.CL cs.AIcited by 30

Not on the map. Why?

Abstract (arXiv)

In the rapidly evolving field of large language models (LLMs), data augmentation (DA) has emerged as a pivotal technique for enhancing model performance by diversifying training examples without the need for additional data collection. This survey explores the transformative impact of LLMs on DA, particularly addressing the unique challenges and opportunities they present in the context of natural language processing (NLP) and beyond. From both data and learning perspectives, we examine various strategies that utilize LLMs for data augmentation, including a novel exploration of learning paradigms where LLM-generated data is used for diverse forms of further training. Additionally, this paper highlights the primary open challenges faced in this domain, ranging from controllable data augmentation to multi-modal data augmentation. This survey highlights a paradigm shift introduced by LLMs in DA, and aims to serve as a comprehensive guide for researchers and practitioners.

Key facts. Cited by 30 papers in the map (data through 14 Oct 2026). Builds on 88 of its 152 references that are in the map. First posted on arXiv 5 Mar 2024.

Builds on

88 of 152 references.

References · 88

  1. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 14,132
  2. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 8,065
  3. 2020Scaling Laws for Neural Language Modelscited by 6,586
  4. 2019BERTScore: Evaluating Text Generation with BERTcited by 5,512
  5. 2022Self-Instruct: Aligning Language Models with Self-Generated Instructionscited by 2,724
  6. 2023Video-LLaMA: An Instruction-tuned Audio-Visual Language Model for Video Understandingcited by 1,471
  7. 2022Few-Shot Parameter-Efficient Fine-Tuning is Better and Cheaper than In-Context Learningcited by 896
  8. 2023ChatGPT Outperforms Crowd-Workers for Text-Annotation Taskscited by 870
  9. 2022Out of One, Many: Using Language Models to Simulate Human Samplescited by 752
  10. 2021BARTScore: Evaluating Generated Text as Text Generationcited by 722
  11. 2023Unifying Large Language Models and Knowledge Graphs: A Roadmapcited by 668
  12. 2023Instruction Tuning for Large Language Models: A Surveycited by 623
38 more
  1. 2021Documenting Large Webtext Corpora: A Case Study on the Colossal Clean Crawled Corpuscited by 492
  2. 2020Efficient Intent Detection with Dual Sentence Encoderscited by 433
  3. 2021A Survey of Data Augmentation Approaches for NLPcited by 409
  4. 2023The Curse of Recursion: Training on Generated Data Makes Models Forgetcited by 356
  5. 2023NLP Evaluation in trouble: On the Need to Measure LLM Data Contamination for each Benchmarkcited by 348
  6. 2022Large Language Models Are Reasoning Teacherscited by 329
  7. 2023Prompting Large Language Model for Machine Translation: A Case Studycited by 250
  8. 2023GPT4Tools: Teaching Large Language Model to Use Tools via Self-instructioncited by 248
  9. 2023In-context Vectors: Making In Context Learning More Effective and Controllable Through Latent Space Steeringcited by 245
  10. 2022Data Contamination: From Memorization to Exploitationcited by 236
  11. 2022Is GPT-3 a Good Data Annotator?cited by 208
  12. 2023Data-centric Artificial Intelligence: A Surveycited by 203
  13. 2021Want To Reduce Labeling Cost? GPT-3 Can Helpcited by 196
  14. 2023Recommender Systems in the Era of Large Language Models (LLMs)cited by 189
  15. 2021GPT3Mix: Leveraging Large-scale Language Models for Text Augmentationcited by 187
  16. 2024Audio Flamingo: A Novel Audio Language Model with Few-Shot Learning and Dialogue Abilitiescited by 186
  17. 2020A Survey on Recent Approaches for Natural Language Processing in Low-Resource Scenarioscited by 171
  18. 2020A Survey on Knowledge Graph-Based Recommender Systemscited by 170
  19. 2023Recommendation as Instruction Following: A Large Language Model Empowered Recommendation Approachcited by 168
  20. 2022M6-Rec: Generative Pretrained Language Models are Open-Ended Recommender Systemscited by 152
  21. 2023What Makes Good Examples for Visual In-Context Learning?cited by 150
  22. 2021A Survey of Graph Neural Networks for Recommender Systems: Challenges, Methods, and Directionscited by 144
  23. 2020Improving Conversational Recommender Systems via Knowledge Graph based Semantic Fusioncited by 122
  24. 2023Automatic Prompt Augmentation and Selection with Chain-of-Thought from Labeled Datacited by 119
  25. 2024Large Language Models Can Learn Temporal Reasoningcited by 111
  26. 2023SCOTT: Self-Consistent Chain-of-Thought Distillationcited by 105
  27. 2023Motif: Intrinsic Motivation from Artificial Intelligence Feedbackcited by 102
  28. 2021An Empirical Survey of Data Augmentation for Limited Data Learning in NLPcited by 101
  29. 2023Diversify Your Vision Datasets with Automatic Diffusion-Based Augmentationcited by 100
  30. 2023CoAnnotating: Uncertainty-Guided Work Allocation between Human and Large Language Models for Data Annotationcited by 94
  31. 2023On Learning to Summarize with Large Language Models as Referencescited by 80
  32. 2022AugESC: Dialogue Augmentation with Large Language Models for Emotional Support Conversationcited by 79
  33. 2021Medically Aware GPT-3 as a Data Generator for Medical Dialogue Summarizationcited by 76
  34. 2023Benchmarking Generation and Evaluation Capabilities of Large Language Models for Instruction Controllable Summarizationcited by 69
  35. 2023ChatBridge: Bridging Modalities with Large Language Model as a Language Catalystcited by 66
  36. 2023Automating Customer Service using LangChain: Building custom open-source GPT Chatbot for organizationscited by 61
  37. 2023T-SciQ: Teaching Multimodal Chain-of-Thought Reasoning via Mixed Large Language Model Signals for Science Question Answeringcited by 60
  38. 2023LatestEval: Addressing Data Contamination in Language Model Evaluation through Dynamic and Time-Sensitive Test Constructioncited by 58

50 most cited of 88 All 88 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 12; 2025: 14; 2026: 4.

30 papers.

Sort:

By area:

  1. 2025Towards Large Reasoning Models: A Survey of Reinforced Reasoning with Large Language Modelscited by 293
  2. 2024A Survey of Large Language Models for Financial Applications: Progress, Prospects and Challengescited by 90
  3. 2025GenEnv: Difficulty-Aligned Co-Evolution Between LLM Agents and Environment Simulatorscited by 42
  4. 2024InternalInspector I2: Robust Confidence Estimation in LLMs through Internal Statescited by 30
  5. 2024Content Moderation by LLM: From Accuracy to Legitimacycited by 30
  6. 2024Generated Data with Fake Privacy: Hidden Dangers of Fine-tuning Large Language Models on Generated Datacited by 22
  7. 2025Evolving LLMs' Self-Refinement Capability via Synergistic Training-Inference Optimizationcited by 16
  8. 2025Scaling Low-Resource MT via Synthetic Data Generation with LLMscited by 14
  9. 2025Recursive Learning Without Collapse: A Weighting-Based Stabilization Frameworkcited by 10
  10. 2026From Brute Force to Semantic Insight: Performance-Guided Data Transformation Design with LLMscited by 7
  11. 2024ToxiLab: How Well Do Open-Source LLMs Generate Synthetic Toxicity Data?cited by 5
  12. 2025Labeling Free-text Data using Language Model Ensemblescited by 5
18 more
  1. 2024Data Diversity as Implicit Regularization: How Does Diversity Shape the Weight Space of Deep Neural Networks?cited by 4
  2. 2025More Data or Better Data? A Critical Analysis of Data Selection and Synthesis for Mathematical Reasoningcited by 4
  3. 2024LlamBERT: Large-scale low-cost data annotation in NLPcited by 3
  4. 2024Towards Ontology-Enhanced Representation Learning for Large Language Modelscited by 3
  5. 2024Are Data Augmentation Methods in Named Entity Recognition Applicable for Uncertainty Estimation?cited by 3
  6. 2024Ontology-Free General-Domain Knowledge Graph-to-Text Generation Dataset Synthesis using Large Language Modelcited by 3
  7. 2025Filtering with Confidence: When Data Augmentation Meets Conformal Predictioncited by 3
  8. 2024Lifelong Event Detection with Embedding Space Separation and Compactioncited by 2
  9. 2024X-ray2CTPA: Leveraging Diffusion Models to Enhance Pulmonary Embolism Classificationcited by 2
  10. 2025Explaining Matters: Leveraging Definitions and Semantic Expansion for Sexism Detectioncited by 2
  11. 2025LLMSynthor: Macro-Aligned Micro-Records Synthesis with Large Language Modelscited by 1
  12. 2025Towards Bridging Review Sparsity in Recommendation with Textual Edge Graph Representationcited by 1
  13. 2026LLM-to-Speech: A Synthetic Data Pipeline for Training Dialectal Text-to-Speech Modelscited by 1
  14. 2025Multi-turn Natural Language to Graph Query Language Translationcited by 0
  15. 2025Accelerating Reinforcement Learning Algorithms Convergence using Pre-trained Large Language Models as Tutors With Advice Reusingcited by 0
  16. 2025Overcoming the Generalization Limits of SLM Finetuning for Shape-Based Extraction of Datatype and Object Propertiescited by 0
  17. 2026Adaptive Data Augmentation with Multi-armed Bandit: Sample-Efficient Embedding Calibration for Implicit Pattern Recognitioncited by 0
  18. 2026Hallucinations in LLMs: A Lifecycle-Based Survey of Causes, Detection, Mitigation, and Preventioncited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2023Rethink the Effectiveness of Text Data Augmentation: An Empirical Analysis
  2. 2024A Survey on Data Synthesis and Augmentation for Large Language Models
  3. 2024A Survey on Data Augmentation in Large Model Era
  4. 2025Multimodal Large Language Models for Image, Text, and Speech Data Augmentation: A Survey
  5. 2024Retrieval-Augmented Data Augmentation for Low-Resource Domain Tasks
  6. 2022Augmenting Interpretable Models with LLMs during Training
  7. 2023A Comprehensive Overview of Large Language Models
  8. 2024Enhancing SLM via ChatGPT and Dataset Augmentation