mlsanityData to 7 Oct 2026

A Note on Shumailov et al. (2024): `AI Models Collapse When Trained on Recursively Generated Data'

arXiv 2410.12954 ↗First posted 16 Oct 2024cs.LG cs.AIcited by 114

Not on the map. Why?

Abstract (arXiv)

The study conducted by Shumailov et al. (2024) demonstrates that repeatedly training a generative model on synthetic data leads to model collapse. This finding has generated considerable interest and debate, particularly given that current models have nearly exhausted the available data. In this work, we investigate the effects of fitting a distribution (through Kernel Density Estimation, or KDE) or a model to the data, followed by repeated sampling from it. Our objective is to develop a theoretical understanding of the phenomenon observed by Shumailov et al. (2024). Our results indicate that the outcomes reported are a statistical phenomenon and may be unavoidable.

Key facts. Cited by 114 papers in the map (data through 7 Oct 2026). Builds on 0 of its 1 references that are in the map. First posted on arXiv 16 Oct 2024.

Builds on

0 of 1 references.

No references here yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 12; 2025: 55; 2026: 47.

114 papers.

Sort:

By area:

  1. 2024One fish, two fish, but not the whole sea: Alignment reduces language models' conceptual diversitycited by 57
  2. 2025Analyzing Uncertainty of LLM-as-a-Judge: Interval Evaluations with Conformal Predictioncited by 26
  3. 2025Multilingual Prompting for Improving LLM Generation Diversitycited by 25
  4. 2024Do Large Language Models Have an English Accent? Evaluating and Improving the Naturalness of Multilingual LLMscited by 19
  5. 2025Rankers, Judges, and Assistants: Towards Understanding the Interplay of LLMs in Information Retrieval Evaluationcited by 15
  6. 2025A solvable model of learning generative diffusion: theory and insightscited by 14
  7. 2025Machine-generated text detection prevents language model collapsecited by 14
  8. 2025Sparse Logit Sampling: Accelerating Knowledge Distillation in LLMscited by 14
  9. 2024Bias Amplification: Large Language Models as Increasingly Biased Mediacited by 12
  10. 2025Do we really have to filter out random noise in pre-training data for language models?cited by 11
  11. 2024Searching for Structure: Investigating Emergent Communication with Large Language Modelscited by 10
  12. 2026From Memorization to Creativity: LLM as a Designer of Novel Neural Architecturescited by 10
38 more
  1. 2025Understanding and Mitigating Bias Inheritance in LLM-based Data Augmentation on Downstream Taskscited by 9
  2. 2025textttDroid: A Resource Suite for AI-Generated Code Detectioncited by 9
  3. 2025WaymoQA: A Multi-View Visual Question Answering Dataset for Safety-Critical Reasoning in Autonomous Drivingcited by 9
  4. 2026Task-Aware LLM Routing with Multi-Level Task-Profile-Guided Data Synthesis for Cold-Start Scenarioscited by 8
  5. 2025GraphGen: Enhancing Supervised Fine-Tuning for LLMs with Knowledge-Driven Synthetic Data Generationcited by 7
  6. 2025Position: LLM Watermarking Should Align Stakeholders' Incentives for Practical Adoptioncited by 7
  7. 2026From Fluent to Verifiable: Claim-Level Auditability for Deep Research Agentscited by 7
  8. 2024Does Few-Shot Learning Help LLM Performance in Code Synthesis?cited by 6
  9. 2025Aleph-Alpha-GermanWeb: Improving German-language LLM pre-training with model-based data curation and synthetic data generationcited by 6
  10. 2025LLM Web Dynamics: Tracing Model Collapse in a Network of LLMscited by 6
  11. 2025WETBench: A Benchmark for Detecting Task-Specific Machine-Generated Text on Wikipediacited by 6
  12. 2024Self-Steering Optimization: Autonomous Preference Optimization for Large Language Modelscited by 5
  13. 2024One world, one opinion? The superstar effect in LLM responsescited by 5
  14. 2025Incident Analysis for AI Agentscited by 5
  15. 2025Bias Amplification in Stable Diffusion's Representation of Stigma Through Skin Tones and Their Homogeneitycited by 5
  16. 2025Controllable Protein Sequence Generation with LLM Preference Optimizationcited by 4
  17. 2026Reactive Writers: How Co-Writing with AI Changes How We Engage with Ideascited by 4
  18. 2025Does Training on Synthetic Data Make Models Less Robust?cited by 3
  19. 2025How large language models judge and influence human cooperationcited by 3
  20. 2025LongTail-Swap: benchmarking language models' abilities on rare wordscited by 3
  21. 2025The Economics of AI Foundation Models: Openness, Competition, and Governancecited by 3
  22. 2025Stabilizing Self-Consuming Diffusion Models with Latent Space Filteringcited by 3
  23. 2026Simulating Complex Multi-Turn Tool Calling Interactions in Stateless Execution Environmentscited by 3
  24. 2024From Test-Taking to Test-Making: Examining LLM Authoring of Commonsense Assessment Itemscited by 2
  25. 2024Hands-On Tutorial: Labeling with LLM and Human-in-the-Loopcited by 2
  26. 2024An information theoretic limit to data amplificationcited by 2
  27. 2025Limits to AI Growth: The Ecological and Social Consequences of Scalingcited by 2
  28. 2025Recursive Training Loops in LLMs: How training data properties modulate distribution shift in generated data?cited by 2
  29. 2025Information Retrieval in the Age of Generative AI: The RGB Modelcited by 2
  30. 2025Dynaword: From One-shot to Continuously Developed Datasetscited by 2
  31. 2025Generative Models for Synthetic Data: Transforming Data Mining in the GenAI Eracited by 2
  32. 2025Learning from the Undesirable: Robust Adaptation of Language Models without Forgettingcited by 2
  33. 2026An Empirical Study on Preference Tuning Generalization and Diversity Under Domain Shiftcited by 2
  34. 2026Knowledge without Wisdom: Measuring Misalignment between LLMs and Intended Impactcited by 2
  35. 2026Parametric Social Identity Injection and Diversification in Public Opinion Simulationcited by 2
  36. 2026AI-Associated Lexical Shifts Across 34 Languages: Cross-Lingual Convergence and Diachronic Uptake in News Writingcited by 2
  37. 2024Incivility and Rigidity: Evaluating the Risks of Fine-Tuning LLMs for Political Argumentationcited by 1
  38. 2025Swallowing the Poison Pills: Insights from Vulnerability Disparity Among LLMscited by 1

50 most cited of 114 All 114 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2026Reviewing Model Collapse and Countermeasures
  2. 2025ForTIFAI: Fending Off Recursive Training Induced Failure for AI Model Collapse
  3. 2025When Models Don't Collapse: On the Consistency of Iterative MLE
  4. 2024Collapse or Thrive? Perils and Promises of Synthetic Data in a Self-Generating World
  5. 2024How Bad is Training on Synthetic Data? A Statistical Analysis of Language Model Collapse
  6. 2024Is Model Collapse Inevitable? Breaking the Curse of Recursion by Accumulating Real and Synthetic Data
  7. 2025A Closer Look at Model Collapse: From a Generalization-to-Memorization Perspective
  8. 2025A Theoretical Perspective: How to Prevent Model Collapse in Self-consuming Training Loops