mlsanityData to 7 Oct 2026

Beyond Model Collapse: Scaling Up with Synthesized Data Requires Verification

arXiv 2406.07515 ↗First posted 11 Jun 2024cs.LG cs.AI stat.MLcited by 43

On the map: Synthetic data and self-improvement for LLMs → Model collapse from synthetic data

Abstract (arXiv)

Large Language Models (LLM) are increasingly trained on data generated by other LLM, either because generated text and images become part of the pre-training corpus, or because synthetized data is used as a replacement for expensive human-annotation. This raises concerns about model collapse, a drop in model performance when their training sets include generated data. Considering that it is easier for both humans and machines to tell between good and bad examples than to generate high-quality samples, we investigate the use of verification on synthesized data to prevent model collapse. We provide a theoretical characterization using Gaussian mixtures, linear classifiers, and linear verifiers to derive conditions with measurable proxies to assess whether the verifier can effectively select synthesized data that leads to optimal performance. We experiment with two practical tasks -- computing matrix eigenvalues with transformers and news summarization with LLMs -- which both exhibit model collapse when trained on generated data, and show that verifiers, even imperfect ones, can indeed be harnessed to prevent model collapse and that our proposed proxy measure strongly correlates with performance.

Key facts. Cited by 43 papers in the map (data through 7 Oct 2026). Placed in Model collapse from synthetic data, one of 13 directions in Synthetic data and self-improvement for LLMs. Builds on 52 of its 67 references that are in the map. First posted on arXiv 11 Jun 2024.

Builds on

52 of 67 references.

Closest ancestors · 5

Same direction, newest first

  1. 2024How Bad is Training on Synthetic Data? A Statistical Analysis of Language Model Collapse7 Apr
  2. 2024Model Collapse Demystified: The Case of Regression12 Feb
  3. 2024A Tale of Tails: Model Collapse as a Change of Scaling Laws10 Feb
  4. 2023Feedback-guided Data Synthesis for Imbalanced Classification29 Sep
  5. 2023The Curse of Recursion: Training on Generated Data Makes Models Forget27 May

Other references · 47

  1. 2017Attention Is All You Needcited by 59,932
  2. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
  3. 2023GPT-4 Technical Reportcited by 18,132
  4. 2022Training language models to follow instructions with human feedbackcited by 15,712
  5. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,865
  6. 2015Distilling the Knowledge in a Neural Networkcited by 10,076
  7. 2023Segment Anythingcited by 8,158
  8. 2020Scaling Laws for Neural Language Modelscited by 6,411
  9. 2022BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and Generationcited by 4,310
  10. 2021Zero-Shot Text-to-Image Generationcited by 4,276
  11. 2022Constitutional AI: Harmlessness from AI Feedbackcited by 2,950
  12. 2022Self-Instruct: Aligning Language Models with Self-Generated Instructionscited by 2,674
35 more
  1. 2022What Can Transformers Learn In-Context? A Case Study of Simple Function Classescited by 700
  2. 2023Instruction Tuning with GPT-4cited by 676
  3. 2024Self-Rewarding Language Modelscited by 654
  4. 2022Grokking: Generalization Beyond Overfitting on Small Algorithmic Datasetscited by 599
  5. 2023Textbooks Are All You Needcited by 561
  6. 2018Born Again Neural Networkscited by 539
  7. 2022Beyond neural scaling laws: beating power law scaling via data pruningcited by 519
  8. 2023TinyStories: How Small Can Language Models Be and Still Speak Coherent English?cited by 425
  9. 2021XL-Sum: Large-Scale Multilingual Abstractive Summarization for 44 Languagescited by 346
  10. 2023Synthetic Data from Diffusion Models Improves ImageNet Classificationcited by 326
  11. 2022Is synthetic data from generative models ready for image recognition?cited by 320
  12. 2020Towards Understanding Ensemble, Knowledge Distillation and Self-Distillation in Deep Learningcited by 292
  13. 2024OpenCodeInterpreter: Integrating Code Generation with Execution and Refinementcited by 242
  14. 2023Self-Consuming Generative Models Go MADcited by 239
  15. 2024A Survey on Data Selection for Language Modelscited by 230
  16. 2024When Scaling Meets LLM Finetuning: The Effect of Data, Model and Finetuning Methodcited by 201
  17. 2023Does Writing with Language Models Reduce Content Diversity?cited by 190
  18. 2020Self-Distillation Amplifies Regularization in Hilbert Spacecited by 171
  19. 2023Diversify Your Vision Datasets with Automatic Diffusion-Based Augmentationcited by 100
  20. 2023On the Stability of Iterative Retraining of Generative Models on their own Datacited by 98
  21. 2022Language Models Can Teach Themselves to Program Bettercited by 75
  22. 2023Large Language Models Suffer From Their Own Output: An Analysis of the Self-Consuming Training Loopcited by 75
  23. 2021Linear algebra with transformerscited by 62
  24. 2022Will Large-scale Generative Models Corrupt Future Datasets?cited by 59
  25. 2023Towards Understanding the Interplay of Generative Artificial Intelligence and the Internetcited by 54
  26. 2023Combining Generative Artificial Intelligence (AI) and the Internet: Heading towards Evolution or Degradation?cited by 41
  27. 2023Don't Play Favorites: Minority Guidance for Diffusion Modelscited by 37
  28. 2024Bootstrapping LLM-based Task-Oriented Dialogue Agents via Self-Talkcited by 34
  29. 2024Self-Correcting Self-Consuming Loops for Generative Model Trainingcited by 34
  30. 2023Learning the greatest common divisor: explaining transformer predictionscited by 33
  31. 2019Distillation ≈ Early Stopping? Harvesting Dark Knowledge Utilizing Anisotropic Information Retrieval For Overparameterized Neural Networkcited by 27
  32. 2023Towards a statistical theory of data selection under weak supervisioncited by 25
  33. 2023Understanding Self-Distillation in the Presence of Label Noisecited by 20
  34. 2022Evaluating Distributional Distortion in Neural Language Modelingcited by 18
  35. 2023Image retrieval outperforms diffusion models on data augmentationcited by 14

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 5; 2025: 18; 2026: 20.

43 papers.

Sort:

By area:

  1. 2024LOKI: A Comprehensive Synthetic Data Detection Benchmark using Large Multimodal Modelscited by 51
  2. 2024How to Synthesize Text Data without Model Collapse?cited by 24
  3. 2025A Closer Look at Model Collapse: From a Generalization-to-Memorization Perspectivecited by 18
  4. 2025A Theoretical Perspective: How to Prevent Model Collapse in Self-consuming Training Loopscited by 15
  5. 2025AgentFrontier: Expanding the Capability Frontier of LLM Agents with ZPD-Guided Data Synthesiscited by 12
  6. 2024Model Collapse in the Self-Consuming Chain of Diffusion Finetuning: A Novel Perspective from Quantitative Trait Modelingcited by 11
  7. 2025Escaping Collapse: The Strength of Weak Data for Large Language Model Trainingcited by 11
  8. 2025When Models Don't Collapse: On the Consistency of Iterative MLEcited by 11
  9. 2025Why Less is More (Sometimes): A Theory of Data Curationcited by 8
  10. 2025Preventing Model Collapse Under Overparametrization: Optimal Mixing Ratios for Interpolation Learning and Ridge Regressioncited by 7
  11. 2025RePro: Training Language Models to Faithfully Recycle the Web for Pretrainingcited by 6
  12. 2025Escaping Model Collapse via Synthetic Data Verification: Near-term Improvements and Long-term Convergencecited by 5
31 more
  1. 2026Language Generation with Replay: A Learning-Theoretic View of Model Collapsecited by 5
  2. 2024Data Diversity as Implicit Regularization: How Does Diversity Shape the Weight Space of Deep Neural Networks?cited by 4
  3. 2025Measuring Lexical Diversity of Synthetic Data Generated through Fine-Grained Persona Promptingcited by 4
  4. 2025Data Can Speak for Itself: Quality-guided Utilization of Wireless Synthetic Datacited by 4
  5. 2025SeDi-Instruct: Enhancing Alignment of Language Models through Self-Directed Instruction Generationcited by 3
  6. 2025Stabilizing Self-Consuming Diffusion Models with Latent Space Filteringcited by 3
  7. 2026A Sustainable AI Economy Needs Data Deals That Work for Generatorscited by 3
  8. 2026Learning from Synthetic Data: Limitations of ERMcited by 3
  9. 2024Learning by Surprise: Adaptive Mitigation of Model Collapse in Large Language Modelscited by 2
  10. 2025Beyond Real Data: Synthetic Data through the Lens of Regularizationcited by 2
  11. 2025Epistemic diversity across language models mitigates knowledge collapsecited by 2
  12. 2026Ambient Dataloops: Generative Models for Dataset Refinementcited by 2
  13. 2025What Matters in LLM-generated Data: Diversity and Its Effect on Model Fine-Tuningcited by 1
  14. 2025Data Value in the Age of Scaling: Understanding LLM Scaling Dynamics Under Real-Synthetic Data Mixturescited by 1
  15. 2026Drift and selection in LLM text ecosystemscited by 1
  16. 2026Model Collapse as Cultural Evolutioncited by 1
  17. 2026Learning from Synthetic Data without Model Collapse in Iterative Instruction Tuningcited by 1
  18. 2025The Anti-Ouroboros Effect: Emergent Resilience in Large Language Models from Recursive Selective Feedbackcited by 0
  19. 2026Can Generative Artificial Intelligence Survive Data Contamination? Theoretical Guarantees under Contaminated Recursive Trainingcited by 0
  20. 2026When and How Human Curation Backfires: Preference Alignment under Multi-Model Self-Consuming Loopcited by 0
  21. 2026Not All Synthetic Data Is Yours to Learn Fromcited by 0
  22. 2026When Sample Selection Bias Precipitates Model Collapsecited by 0
  23. 2026The Oligarch Barely Steers Model Collapse in Multi-Model Ecosystemscited by 0
  24. 2026Break Step: Recursive Training Resonates with Replayed Sampling Noisecited by 0
  25. 2026SynthSentry: Detecting Synthetic Data Contamination in Language Model Training Datacited by 0
  26. 2026NeMo Data Designer: An Extensible Framework for Multimodal Synthetic Data Generationcited by 0
  27. 2026The evolution of sex for artificial intelligence: a population-genetic framework for multigenerational model populationscited by 0
  28. 2026Preventing Model Collapse: A Fisher-Rao Perspective on the Dynamics of Training with Synthetic Datacited by 0
  29. 2026Improving Generative Model Self-Training with Geometrically Modified Outputscited by 0
  30. 2026It's All Training: A Fully Synthetic Single-Stage Recipe for LLMscited by 0
  31. 2026No Model Required: Text Entropy Rate Filtering Mitigates Iterative Fine-Tuning Collapsecited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Machine-generated text detection prevents language model collapse
  2. 2024Strong Model Collapse
  3. 2024Is Model Collapse Inevitable? Breaking the Curse of Recursion by Accumulating Real and Synthetic Data
  4. 2025Scaling Laws of Synthetic Data for Language Models
  5. 2024Forewarned is Forearmed: Leveraging LLMs for Data Synthesis through Failure-Inducing Exploration
  6. 2025Demystifying Synthetic Data in LLM Pre-training: A Systematic Study of Scaling Laws, Benefits, and Pitfalls
  7. 2026CAVEWOMAN: How Large Language Models Behave Under Linguistic Input and Output Compression
  8. 2026A Survey on Evaluating Quality and Trustworthiness in LLM-Generated Data