mlsanityData to 7 Oct 2026

LLM synthetic data generation

275 papers2 this weekMedium confidence in this grouping: mostly one topic, with blurry edgesMedium confidence in this grouping: mostly one topic, with blurry edges · About

Keywords: synthetic, llms, language, generation, large

Key facts. LLM synthetic data generation: 275 papers on arXiv since 2021 (data through 7 Oct 2026). 29 of its 275 mapped papers (11%) cite “On LLMs-Driven Synthetic Data Generation, Curation, and Evaluation: A Survey” (2024, arXiv 2406.15126). 22 of its 275 mapped papers (8%) cite “Best Practices and Lessons Learned on Synthetic Data” (2024, arXiv 2404.07503). 5 new papers 17 Sep–7 Oct.

What’s new

17 Sep – 7 Oct 2026

5 new papers, 17 Sep – 7 Oct; 2 this week.

  1. This week · 2
  2. Coverage, Not Difficulty, Sets How Much Synthetic Data an Activation Probe Needs
  3. Improving Synthetic Data Generation for Argument Mining via Adversarial Reinforcement Learning
  4. Earlier · 3
  5. Training-Aware Target Coverage for Synthetic Data Selection
  6. Synthetic Data Characterization via Training Dynamics also fits: Model collapse from synthetic data
  7. DataCanvas-EDU: An Agentic Framework for Instructor-Guided Synthetic Data Generation in Business Analytics Education

Foundations

Most cited by papers here

  1. On LLMs-Driven Synthetic Data Generation, Curation, and Evaluation: A Survey2024cited by 29 here
  2. Best Practices and Lessons Learned on Synthetic Data2024cited by 22 here
  3. Scaling Synthetic Data Creation with 1,000,000,000 Personas2024cited by 18 here
  4. Synthetic Data Generation with Large Language Models for Text Classification: Potential and Limitations2023cited by 16 here
  5. On the Diversity of Synthetic Data and its Impact on Training Large Language Models2024cited by 9 here

Lineage

The papers this direction builds on, oldest first.

Each paper on the line cites the one before.

  1. 2023The Curse of Recursion: Training on Generated Data Makes Models Forgetcited by 17 here
  2. 2023Reinforced Self-Training (ReST) for Language Modelingcited by 7 here
  3. 2023Beyond Human Data: Scaling Self-Training for Problem-Solving with Language Modelscited by 6 here
  4. 2024Self-Play Fine-Tuning Converts Weak Language Models to Strong Language Modelscited by 8 here
  5. 2024Self-Rewarding Language Modelscited by 8 here
  6. 2024Best Practices and Lessons Learned on Synthetic Datacited by 22 here
  7. 2024On LLMs-Driven Synthetic Data Generation, Curation, and Evaluation: A Surveycited by 29 here
  8. 2024On the Diversity of Synthetic Data and its Impact on Training Large Language Modelscited by 9 here
16 more it builds on, not in this chain
  1. 2022STaR: Bootstrapping Reasoning With Reasoningcited by 7 here
  2. 2022Large Language Models Can Self-Improvecited by 12 here
  3. 2022Self-Instruct: Aligning Language Models with Self-Generated Instructionscited by 60 here
  4. 2023Does Synthetic Data Generation of LLMs Help Clinical Text Mining?cited by 10 here
  5. 2023MetaMath: Bootstrap Your Own Mathematical Questions for Large Language Modelscited by 15 here
  6. 2023Synthetic Data Generation with Large Language Models for Text Classification: Potential and Limitationscited by 16 here
  7. 2024A Tale of Tails: Model Collapse as a Change of Scaling Lawscited by 10 here
  8. 2024Synthetic Data (Almost) from Scratch: Generalized Instruction Tuning for Language Modelscited by 9 here
  9. 2024Is Model Collapse Inevitable? Breaking the Curse of Recursion by Accumulating Real and Synthetic Datacited by 8 here
  10. 2024How Bad is Training on Synthetic Data? A Statistical Analysis of Language Model Collapsecited by 7 here
  11. 2024CodecLM: Aligning Language Models with Tailored Synthetic Datacited by 7 here
  12. 2024Better Synthetic Data by Retrieving and Transforming Existing Datasetscited by 6 here
  13. 2024Beyond Model Collapse: Scaling Up with Synthesized Data Requires Verificationcited by 8 here
  14. 2024Magpie: Alignment Data Synthesis from Scratch by Prompting Aligned LLMs with Nothingcited by 20 here
  15. 2024Scaling Synthetic Data Creation with 1,000,000,000 Personascited by 18 here
  16. 2024Evaluating Language Models as Synthetic Data Generatorscited by 6 here

The 24 most cited of 151.

Browse

30 of 275, ranked 6–35

  1. 6Synthetic Data Generation Using Large Language Models: Advances in Text and Code20256
  2. 7Better Synthetic Data by Retrieving and Transforming Existing Datasets20246
  3. 8Evaluating Language Models as Synthetic Data Generators20246
  4. 9Generative AI for Synthetic Data Generation: Methods, Challenges and the Future20245
  5. 10Condor: Enhance LLM Alignment with Knowledge-Driven Data Synthesis and Refinement20254
  6. 11DataDreamer: A Tool for Synthetic Data Generation and Reproducible LLM Workflows20244
  7. 12Efficacy of Synthetic Data as a Benchmark20244
  8. 13Generating Faithful Synthetic Data with Large Language Models: A Case Study in Computational Social Science20234
  9. 14Measuring Diversity in Synthetic Datasets20253
  10. 15DataGen: Unified Synthetic Dataset Generation via Large Language Models20243
All 30
  1. 16A Survey on Data Synthesis and Augmentation for Large Language Models20243
  2. 17Not All LLM-Generated Data Are Equal: Rethinking Data Weighting in Text Classification20243
  3. 18The False Promise of Imitating Proprietary LLMs20233
  4. 19Simple synthetic data reduces sycophancy in large language models20233
  5. 20BARE: Leveraging Base Language Models for Few-Shot Synthetic Data Generation20252
  6. 21Scaling Laws of Synthetic Data for Language Models20252
  7. 22Scaling Low-Resource MT via Synthetic Data Generation with LLMs20252
  8. 23Demystifying Synthetic Data in LLM Pre-training: A Systematic Study of Scaling Laws, Benefits, and Pitfalls20252
  9. 24Unveiling the Flaws: Exploring Imperfections in Synthetic Data and Mitigation Strategies for Large Language Models20242
  10. 25Quality Matters: Evaluating Synthetic Data for Tool-Using LLMs20242
  11. 26Surveying the Effects of Quality, Diversity, and Complexity in Synthetic Data From Large Language Models20242
  12. 27How to Synthesize Text Data without Model Collapse?20242
  13. 28DSPy: Compiling Declarative Language Model Calls into Self-Improving Pipelines20232
  14. 29Better as Generators Than Classifiers: Leveraging LLMs and Synthetic Data for Low-Resource Multilingual Classification20261
  15. 30On Synthetic Data Strategies for Domain-Specific Generative Retrieval20251
  16. 31Delving into: the quantification of Ai-generated content on the internet (synthetic data)20251
  17. 32SynLexLM: Scaling Legal LLMs with Synthetic Data and Curriculum Learning20251
  18. 33Towards Understanding Bias in Synthetic Data for Evaluation20251
  19. 34Valid Inference with Imperfect Synthetic Data20251
  20. 35BeyondWeb: Lessons from Scaling Synthetic Data for Trillion-scale Pretraining20251