mlsanityData to 14 Oct 2026

Reviewing Model Collapse and Countermeasures

arXiv 2608.21366 ↗First posted 17 Jun 2026cs.AI cs.LGcited by 0

On the map: Synthetic data and self-improvement for LLMs → Model collapse from synthetic data

Abstract (arXiv)

Driven by massive amounts of web-scale data, generative AI (GenAI) has achieved remarkable progress, enabling various applications in diverse sectors. The advances of GenAI have actuated practitioners to use AI-synthesized data for training next-generation AI models. Undeniably, using synthetic data has alleviated the increasing stringent demand for data supply. Unfortunately, it also introduces a new critical issue: in a self-consuming cycle between model and data, the model ultimately collapse, raising more trustworthiness concerns to GenAI. In recent years, increasingly more studies have investigated the phenomenon of model collapse (MC) and explored potential solutions to mitigate it. However, the review of the phenomenon of MC still remains blank. To fill this gap, this paper provides an up-to-date overview of these studies for consolidating and reviewing the progress of MC in different application scenarios and countermeasures for mitigating MC. We also highlight challenges and future research opportunities.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Model collapse from synthetic data, one of 13 directions in Synthetic data and self-improvement for LLMs. Builds on 35 of its 50 references that are in the map. First posted on arXiv 17 Jun 2026.

Builds on

35 of 50 references.

Closest ancestors · 7

Same direction, newest first

  1. 2025Multi-modal Synthetic Data Training and Model Collapse: Insights from VLMs and Diffusion Models10 May
  2. 2024Analyzing and Mitigating Model Collapse in Rectified Flow Models11 Dec
  3. 2024Strong Model Collapse7 Oct
  4. 2024How Bad is Training on Synthetic Data? A Statistical Analysis of Language Model Collapse7 Apr
  5. 2024Model Collapse Demystified: The Case of Regression12 Feb
  6. 2024A Tale of Tails: Model Collapse as a Change of Scaling Laws10 Feb
  7. 2023The Curse of Recursion: Training on Generated Data Makes Models Forget27 May

Other references · 28

  1. 2017Attention Is All You Needcited by 60,361
  2. 2020Denoising Diffusion Probabilistic Modelscited by 21,076
  3. 2015Distilling the Knowledge in a Neural Networkcited by 10,324
  4. 2021Training Verifiers to Solve Math Word Problemscited by 10,109
  5. 2020Scaling Laws for Neural Language Modelscited by 6,586
  6. 2016Overcoming catastrophic forgetting in neural networkscited by 4,787
  7. 2021Zero-Shot Text-to-Image Generationcited by 4,288
  8. 2022Elucidating the Design Space of Diffusion-Based Generative Modelscited by 3,045
  9. 2017Gradient Episodic Memory for Continual Learningcited by 1,898
  10. 2017High-Resolution Image Synthesis and Semantic Manipulation with Conditional GANscited by 1,179
  11. 2019Machine Unlearningcited by 1,061
  12. 2019Decoupling Representation and Classifier for Long-Tailed Recognitioncited by 809
16 more
  1. 2020Prevalence of Neural Collapse during the terminal phase of deep learning trainingcited by 719
  2. 2023VideoPoet: A Large Language Model for Zero-Shot Video Generationcited by 492
  3. 2017Regularizing Neural Networks by Penalizing Confident Output Distributionscited by 464
  4. 2018Model compression via distillation and quantizationcited by 289
  5. 2023Self-Consuming Generative Models Go MADcited by 250
  6. 2022Will we run out of data? Limits of LLM scaling based on human-generated datacited by 195
  7. 2021Neural Collapse Under MSE Loss: Proximity to and Dynamics on the Central Pathcited by 158
  8. 2020MetaPoison: Practical General-purpose Clean-label Data Poisoningcited by 120
  9. 2024Consistency Flow Matching: Defining Straight Flows with Velocity Consistencycited by 104
  10. 2023On the Stability of Iterative Retraining of Generative Models on their own Datacited by 100
  11. 2025Subliminal Learning: Language models transmit behavioral traits via hidden signals in datacited by 95
  12. 2022Wild Patterns Reloaded: A Survey of Machine Learning Security against Training Data Poisoningcited by 94
  13. 2022Will Large-scale Generative Models Corrupt Future Datasets?cited by 59
  14. 2022CodeFill: Multi-token Code Completion by Jointly Learning from Structure and Naming Sequencescited by 50
  15. 2022Hidden Poison: Machine Unlearning Enables Camouflaged Poisoning Attackscited by 43
  16. 2024LaWa: Using Latent Space for In-Generation Image Watermarkingcited by 28

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2024A Note on Shumailov et al. (2024): `AI Models Collapse When Trained on Recursively Generated Data'
  2. 2025ForTIFAI: Fending Off Recursive Training Induced Failure for AI Model Collapse
  3. 2025Position: Model Collapse Does Not Mean What You Think
  4. 2024Collapse or Thrive? Perils and Promises of Synthetic Data in a Self-Generating World
  5. 2025When Models Don't Collapse: On the Consistency of Iterative MLE
  6. 2024Is Model Collapse Inevitable? Breaking the Curse of Recursion by Accumulating Real and Synthetic Data
  7. 2025Escaping Model Collapse via Synthetic Data Verification: Near-term Improvements and Long-term Convergence
  8. 2025A Closer Look at Model Collapse: From a Generalization-to-Memorization Perspective