# Comprehensive Exploration of Synthetic Data Generation: A Survey

arXiv: https://arxiv.org/abs/2401.02524 · HTML: https://mlsanity.com/paper/2401.02524/

## Key facts

- Cited by 60 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 4 Jan 2024.

## Built on by (most cited first, 30 of 60)

- [Scaling Synthetic Data Creation with 1,000,000,000 Personas](https://mlsanity.com/paper/2406.20094/) (2024)
- [ToolACE: Winning the Points of LLM Function Calling](https://mlsanity.com/paper/2409.00920/) (2024)
- [Scaling Laws for Precision](https://mlsanity.com/paper/2411.04330/) (2024)
- [On the Diversity of Synthetic Data and its Impact on Training Large Language Models](https://mlsanity.com/paper/2410.15226/) (2024)
- [Surveying the Effects of Quality, Diversity, and Complexity in Synthetic Data From Large Language Models](https://mlsanity.com/paper/2412.02980/) (2024)
- [Generative AI for Synthetic Data Generation: Methods, Challenges and the Future](https://mlsanity.com/paper/2403.04190/) (2024)
- [ClinicalBench: Can LLMs Beat Traditional ML Models in Clinical Prediction?](https://mlsanity.com/paper/2411.06469/) (2024)
- [Montessori-Instruct: Generate Influential Training Data Tailored for Student Learning](https://mlsanity.com/paper/2410.14208/) (2024)
- [ToolForge: A Data Synthesis Pipeline for Multi-Hop Search without Real-World APIs](https://mlsanity.com/paper/2512.16149/) (2025)
- [DiffLM: Controllable Synthetic Data Generation via Diffusion Language Models](https://mlsanity.com/paper/2411.03250/) (2024)
- [Towards Building Specialized Generalist AI with System 1 and System 2 Fusion](https://mlsanity.com/paper/2407.08642/) (2024)
- [From Hypothesis to Publication: A Comprehensive Survey of AI-Driven Research Support Systems](https://mlsanity.com/paper/2503.01424/) (2025)
- [A Survey of Inductive Reasoning for Large Language Models](https://mlsanity.com/paper/2510.10182/) (2025)
- [Beyond Real Faces: Synthetic Datasets Can Achieve Reliable Recognition Performance without Privacy Compromise](https://mlsanity.com/paper/2510.17372/) (2025)
- [Logic Unseen: Revealing the Logical Blindspots of Vision-Language Models](https://mlsanity.com/paper/2508.11317/) (2025)
- [Tabular Data Synthesis with Differential Privacy: A Survey](https://mlsanity.com/paper/2411.03351/) (2024)
- [Self-Improvement in Multimodal Large Language Models: A Survey](https://mlsanity.com/paper/2510.02665/) (2025)
- [Mitigating Sycophancy in Decoder-Only Transformer Architectures: Synthetic Data Intervention](https://mlsanity.com/paper/2411.10156/) (2024)
- [Deep Learning in Palmprint Recognition-A Comprehensive Survey](https://mlsanity.com/paper/2501.01166/) (2025)
- [A survey of datasets for computer vision in agriculture](https://mlsanity.com/paper/2502.16950/) (2025)
- [Enterprise Large Language Model Evaluation Benchmark](https://mlsanity.com/paper/2506.20274/) (2025)
- [Synthetic Data Generation by Supervised Neural Gas Network for Physiological Emotion Recognition Data](https://mlsanity.com/paper/2501.16353/) (2025)
- [Synthetic Artifact Auditing: Tracing LLM-Generated Synthetic Data Usage in Downstream Applications](https://mlsanity.com/paper/2502.00808/) (2025)
- [Synthetic Data Generation of Body Motion Data by Neural Gas Network for Emotion Recognition](https://mlsanity.com/paper/2503.14513/) (2025)
- [Feeding LLM Annotations to BERT Classifiers at Your Own Risk](https://mlsanity.com/paper/2504.15432/) (2025)
- [Code-driven Number Sequence Calculation: Enhancing the inductive Reasoning Abilities of Large Language Models](https://mlsanity.com/paper/2510.14620/) (2025)
- [Uncertainty Quantification and Data Efficiency in AI: An Information-Theoretic Perspective](https://mlsanity.com/paper/2512.05267/) (2025)
- [GR-SAP: Generative Replay for Safety Alignment Preservation during Fine-Tuning](https://mlsanity.com/paper/2603.10243/) (2026)
- [Code-Driven Inductive Synthesis: Enhancing Reasoning Abilities of Large Language Models with Sequences](https://mlsanity.com/paper/2503.13109/) (2025)
- [PuckTrick: A Library for Making Synthetic Data More Realistic](https://mlsanity.com/paper/2506.18499/) (2025)

## Abstract (arXiv)

Recent years have witnessed a surge in the popularity of Machine Learning (ML), applied across diverse domains. However, progress is impeded by the scarcity of training data due to expensive acquisition and privacy legislation. Synthetic data emerges as a solution, but the abundance of released models and limited overview literature pose challenges for decision-making. This work surveys 417 Synthetic Data Generation (SDG) models over the last decade, providing a comprehensive overview of model types, functionality, and improvements. Common attributes are identified, leading to a classification and trend analysis. The findings reveal increased model performance and complexity, with neural network-based approaches prevailing, except for privacy-preserving data generation. Computer vision dominates, with GANs as primary generative models, while diffusion models, transformers, and RNNs compete. Implications from our performance evaluation highlight the scarcity of common metrics and datasets, making comparisons challenging. Additionally, the neglect of training and computational costs in literature necessitates attention in future research. This work serves as a guide for SDG model selection and identifies crucial areas for future exploration.