mlsanityData to 7 Oct 2026

Synthetic Data Generation with Large Language Models for Text Classification: Potential and Limitations

arXiv 2310.07849 ↗First posted 11 Oct 2023cs.CL cs.AIcited by 96

On the map: Synthetic data and self-improvement for LLMs → LLM synthetic data generation

Abstract (arXiv)

The collection and curation of high-quality training data is crucial for developing text classification models with superior performance, but it is often associated with significant costs and time investment. Researchers have recently explored using large language models (LLMs) to generate synthetic datasets as an alternative approach. However, the effectiveness of the LLM-generated synthetic data in supporting model training is inconsistent across different classification tasks. To better understand factors that moderate the effectiveness of the LLM-generated synthetic data, in this study, we look into how the performance of models trained on these synthetic data may vary with the subjectivity of classification. Our results indicate that subjectivity, at both the task level and instance level, is negatively associated with the performance of the model trained on synthetic data. We conclude by discussing the implications of our work on the potential and limitations of leveraging LLM for synthetic data generation.

Key facts. Cited by 96 papers in the map (data through 7 Oct 2026). Placed in LLM synthetic data generation, one of 13 directions in Synthetic data and self-improvement for LLMs. Builds on 32 of its 59 references that are in the map. First posted on arXiv 11 Oct 2023.

Builds on

32 of 59 references.

References · 32

  1. 2017Attention Is All You Needcited by 59,932
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  3. 2020Language Models are Few-Shot Learnerscited by 32,199
  4. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,266
  5. 2018A Style-Based Generator Architecture for Generative Adversarial Networkscited by 5,986
  6. 2021Finetuned Language Models Are Zero-Shot Learnerscited by 3,747
  7. 2021GLIDE: Towards Photorealistic Image Generation and Editing with Text-Guided Diffusion Modelscited by 3,595
  8. 2015Character-level Convolutional Networks for Text Classificationcited by 2,293
  9. 2020GoEmotions: A Dataset of Fine-Grained Emotionscited by 479
  10. 2022ToxiGen: A Large-Scale Machine-Generated Dataset for Adversarial and Implicit Hate Speech Detectioncited by 419
  11. 2022Is synthetic data from generative models ready for image recognition?cited by 320
  12. 2013Good Debt or Bad Debt: Detecting Semantic Orientations in Economic Textscited by 310
20 more
  1. 2021All That's 'Human' Is Not Gold: Evaluating Human Evaluation of Generated Textcited by 287
  2. 2021DatasetGAN: Efficient Labeled Data Factory with Minimal Human Effortcited by 264
  3. 2021Annotators with Attitudes: How Annotator Beliefs And Identities Bias Toxic Language Detectioncited by 258
  4. 2022ZeroGen: Efficient Zero-shot Learning via Dataset Generationcited by 209
  5. 2022Generating Training Data with Language Models: Towards Zero-Shot Language Understandingcited by 199
  6. 2021GPT3Mix: Leveraging Large-scale Language Models for Text Augmentationcited by 185
  7. 2020Data Augmentation using Pre-trained Transformer Modelscited by 168
  8. 2023Does Synthetic Data Generation of LLMs Help Clinical Text Mining?cited by 151
  9. 2023Increasing Diversity While Maintaining Accuracy: Text Data Generation with Large Language Models and Human Interventionscited by 121
  10. 2022Jury Learning: Integrating Dissenting Voices into Machine Learning Modelscited by 115
  11. 2019FewRel 2.0: Towards More Challenging Few-Shot Relation Classificationcited by 94
  12. 2021Is GPT-3 Text Indistinguishable from Human Text? Scarecrow: A Framework for Scrutinizing Machine Textcited by 89
  13. 2021Towards Zero-Label Language Learningcited by 76
  14. 2022Is Your Toxicity My Toxicity? Exploring the Impact of Rater Identity on Toxicity Annotationcited by 67
  15. 2019This dataset does not exist: training models from generated imagescited by 65
  16. 2023On the Creativity of Large Language Modelscited by 57
  17. 2022Data Augmentation for Intent Classification with Off-the-shelf Large Language Modelscited by 49
  18. 2022Weakly Supervised Data Augmentation Through Prompting for Dialogue Understandingcited by 32
  19. 2023On the Design of AI-powered Code Assistants for Notebookscited by 28
  20. 2021Directed Diversity: Leveraging Language Embedding Distances for Collective Creativity in Crowd Ideationcited by 23

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 1; 2024: 33; 2025: 45; 2026: 17.

96 papers.

Sort:

By area:

  1. 2024RAGBench: Explainable Benchmark for Retrieval-Augmented Generation Systemscited by 115
  2. 2025Synthetic Data Generation Using Large Language Models: Advances in Text and Codecited by 61
  3. 2024Is In-Context Learning in Large Language Models Bayesian? A Martingale Perspectivecited by 51
  4. 20251.4 Million Open-Source Distilled Reasoning Dataset to Empower Large Language Model Trainingcited by 49
  5. 2024Can LLMs Replace Manual Annotation of Software Engineering Artifacts?cited by 40
  6. 2024MarS: a Financial Market Simulation Engine Powered by Generative Foundation Modelcited by 27
  7. 2025Knowledge Distillation and Dataset Distillation of Large Language Models: Emerging Trends, Challenges, and Future Directionscited by 25
  8. 2024Adaptive Ensembles of Fine-Tuned Transformers for LLM-Generated Text Detectioncited by 22
  9. 2024Federated Large Language Models: Current Progress and Future Directionscited by 22
  10. 2023TarGEN: Targeted Data Generation with Large Language Modelscited by 17
  11. 2025JITServe: SLO-aware LLM Serving with Imprecise Request Informationcited by 17
  12. 2024Differentially Private Tabular Data Synthesis using Large Language Modelscited by 16
38 more
  1. 2024The Power of LLM-Generated Synthetic Data for Stance Detection in Online Political Discussionscited by 16
  2. 2024Towards a Theoretical Understanding of Synthetic Data in LLM Post-Training: A Reverse-Bottleneck Perspectivecited by 15
  3. 2025A Token is Worth over 1,000 Tokens: Efficient Knowledge Distillation through Low-Rank Clonecited by 15
  4. 2024CTSyn: A Foundation Model for Cross Tabular Data Generationcited by 13
  5. 2024Unveiling the Flaws: Exploring Imperfections in Synthetic Data and Mitigation Strategies for Large Language Modelscited by 13
  6. 2024The New Agronomists: Language Models are Experts in Crop Managementcited by 12
  7. 2024Reason from Fallacy: Enhancing Large Language Models' Logical Reasoning through Logical Fallacy Understandingcited by 12
  8. 2025Distributed LLMs and Multimodal Large Language Models: A Survey on Advances, Challenges, and Future Directionscited by 10
  9. 2024DiffLM: Controllable Synthetic Data Generation via Diffusion Language Modelscited by 9
  10. 2025Synthetic Text Generation for Training Large Language Models via Gradient Matchingcited by 9
  11. 2024A Multi-Faceted Evaluation Framework for Assessing Synthetic Data Generated by Large Language Modelscited by 8
  12. 2024Evaluating Explanations Through LLMs: Beyond Traditional User Studiescited by 8
  13. 2024Synthetic Data Generation with Large Language Models for Personalized Community Question Answeringcited by 8
  14. 2025WhatELSE: Shaping Narrative Spaces at Configurable Level of Abstraction for AI-bridged Interactive Storytellingcited by 8
  15. 2024FM2DS: Few-Shot Multimodal Multihop Data Synthesis with Knowledge Distillation for Question Answeringcited by 7
  16. 2024Not All LLM-Generated Data Are Equal: Rethinking Data Weighting in Text Classificationcited by 6
  17. 2025Automatic Dataset Generation for Knowledge Intensive Question Answering Taskscited by 6
  18. 2024Can Code-Switched Texts Activate a Knowledge Switch in LLMs? A Case Study on English-Korean Code-Switchingcited by 5
  19. 2024Mitigating Sycophancy in Decoder-Only Transformer Architectures: Synthetic Data Interventioncited by 5
  20. 2025Knowledge Hierarchy Guided Biological-Medical Dataset Distillation for Domain LLM Trainingcited by 5
  21. 2025Measuring Diversity in Synthetic Datasetscited by 5
  22. 2025Standardizing Intelligence: Aligning Generative AI for Regulatory and Operational Compliancecited by 5
  23. 2024Learning Program Behavioral Models from Synthesized Input-Output Pairscited by 4
  24. 2024An Evaluation of Standard Statistical Models and LLMs on Time Series Forecastingcited by 4
  25. 2024Enhancing Tourism Recommender Systems for Sustainable City Trips Using Retrieval-Augmented Generationcited by 4
  26. 2025Synthetic Artifact Auditing: Tracing LLM-Generated Synthetic Data Usage in Downstream Applicationscited by 4
  27. 2024Aligning Actions and Walking to LLM-Generated Textual Descriptionscited by 3
  28. 2024Learning Invariant Causal Mechanism from Vision-Language Modelscited by 3
  29. 2025Low-resource Information Extraction with the European Clinical Case Corpuscited by 3
  30. 2025Bridging Generative and Discriminative Learning: Few-Shot Relation Extraction via Two-Stage Knowledge-Guided Pre-trainingcited by 3
  31. 2025A Scoping Review of Synthetic Data Generation by Language Models in Biomedical Research and Application: Data Utility and Quality Perspectivescited by 3
  32. 2025ChemOrch: Empowering LLMs with Chemical Intelligence via Synthetic Instructionscited by 3
  33. 2025ssToken: Self-modulated and Semantic-aware Token Selection for LLM Fine-tuningcited by 3
  34. 2025Polypersona: Persona-Grounded LLM for Synthetic Survey Responsescited by 3
  35. 2024Selective Fine-tuning on LLM-labeled Data May Reduce Reliance on Human Annotation: A Case Study Using Schedule-of-Event Table Detectioncited by 2
  36. 2025Artificial Conversations, Real Results: Fostering Language Detection with Synthetic Datacited by 2
  37. 2025Feeding LLM Annotations to BERT Classifiers at Your Own Riskcited by 2
  38. 2025Hierarchical Level-Wise News Article Clustering via Multilingual Matryoshka Embeddingscited by 2

50 most cited of 96 All 96 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2024Efficacy of Synthetic Data as a Benchmark
  2. 2023Synthetic Data Generation in Low-Resource Settings via Fine-Tuning of Large Language Models
  3. 2026Better as Generators Than Classifiers: Leveraging LLMs and Synthetic Data for Low-Resource Multilingual Classification
  4. 2024A Survey on Data Synthesis and Augmentation for Large Language Models
  5. 2025Demystifying Synthetic Data in LLM Pre-training: A Systematic Study of Scaling Laws, Benefits, and Pitfalls
  6. 2023Generating Faithful Synthetic Data with Large Language Models: A Case Study in Computational Social Science
  7. 2024On the Diversity of Synthetic Data and its Impact on Training Large Language Models
  8. 2025Scaling Laws of Synthetic Data for Language Models