mlsanityData to 14 Oct 2026

The Curse of Recursion: Training on Generated Data Makes Models Forget

arXiv 2305.17493 ↗First posted 27 May 2023cs.LG cs.AI cs.CLcited by 356

On the map: Diffusion models → Discrete and text diffusion

also in 1 more
Abstract (arXiv)

Stable Diffusion revolutionised image creation from descriptive text. GPT-2, GPT-3(.5) and GPT-4 demonstrated astonishing performance across a variety of language tasks. ChatGPT introduced such language models to the general public. It is now clear that large language models (LLMs) are here to stay, and will bring about drastic change in the whole ecosystem of online text and images. In this paper we consider what the future might hold. What will happen to GPT-{n} once LLMs contribute much of the language found online? We find that use of model-generated content in training causes irreversible defects in the resulting models, where tails of the original content distribution disappear. We refer to this effect as Model Collapse and show that it can occur in Variational Autoencoders, Gaussian Mixture Models and LLMs. We build theoretical intuition behind the phenomenon and portray its ubiquity amongst all learned generative models. We demonstrate that it has to be taken seriously if we are to sustain the benefits of training from large-scale data scraped from the web. Indeed, the value of data collected about genuine human interactions with systems will be increasingly valuable in the presence of content generated by LLMs in data crawled from the Internet.

Key facts. Cited by 356 papers in the map (data through 14 Oct 2026). Placed in Discrete and text diffusion, one of 14 directions in Diffusion models. Builds on 19 of its 30 references that are in the map. First posted on arXiv 27 May 2023.

Builds on

19 of 30 references.

References · 19

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  2. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  3. 2020Language Models are Few-Shot Learnerscited by 32,419
  4. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,372
  5. 2016Overcoming catastrophic forgetting in neural networkscited by 4,787
  6. 2013Auto-Encoding Variational Bayescited by 4,053
  7. 2022OPT: Open Pre-trained Transformer Language Modelscited by 3,512
  8. 2016Learning without Forgettingcited by 2,496
  9. 2019Energy and Policy Considerations for Deep Learning in NLPcited by 1,254
  10. 2017BadNets: Identifying Vulnerabilities in the Machine Learning Model Supply Chaincited by 1,106
  11. 2017Memory Aware Synapses: Learning what (not) to forgetcited by 1,105
  12. 2014Deep Neural Networks are Easily Fooled: High Confidence Predictions for Unrecognizable Imagescited by 776
7 more
  1. 2019CTRL: A Conditional Transformer Language Model for Controllable Generationcited by 762
  2. 2019Three scenarios for continual learningcited by 614
  3. 2019Release Strategies and the Social Impacts of Language Modelscited by 513
  4. 2023Poisoning Web-Scale Training Datasets is Practicalcited by 290
  5. 2018Task-Free Continual Learningcited by 221
  6. 2021Poisoning and Backdooring Contrastive Learningcited by 151
  7. 2020Sponge Examples: Energy-Latency Attacks on Neural Networkscited by 111

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 52; 2024: 114; 2025: 96; 2026: 94.

356 papers.

Sort:

By area:

  1. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 13,028
  2. 2023Reinforced Self-Training (ReST) for Language Modelingcited by 500
  3. 2023Weak-to-Strong Generalization: Eliciting Strong Capabilities With Weak Supervisioncited by 467
  4. 2024Scaling Synthetic Data Creation with 1,000,000,000 Personascited by 417
  5. 2024HuatuoGPT-o1, Towards Medical Complex Reasoning with LLMscited by 265
  6. 2023Self-Consuming Generative Models Go MADcited by 250
  7. 2023On the Reliability of Watermarks for Large Language Modelscited by 189
  8. 2025A Survey of Reinforcement Learning for Large Reasoning Modelscited by 180
  9. 2024Is Model Collapse Inevitable? Breaking the Curse of Recursion by Accumulating Real and Synthetic Datacited by 151
  10. 2024O1 Replication Journey: A Strategic Progress Report -- Part 1cited by 143
  11. 2024VidProM: A Million-scale Real Prompt-Gallery Dataset for Text-to-Video Diffusion Modelscited by 131
  12. 2024A Tale of Tails: Model Collapse as a Change of Scaling Lawscited by 130
38 more
  1. 2023Zhongjing: Enhancing the Chinese Medical Capabilities of Large Language Model through Expert Feedback and Real-world Multi-turn Dialoguecited by 119
  2. 2024Mapping the Increasing Use of LLMs in Scientific Paperscited by 118
  3. 2023Artificial Artificial Artificial Intelligence: Crowd Workers Widely Use Large Language Models for Text Production Taskscited by 103
  4. 2023On the Stability of Iterative Retraining of Generative Models on their own Datacited by 100
  5. 2023The Curious Decline of Linguistic Diversity: Training Language Models on Synthetic Textcited by 100
  6. 2024RL on Incorrect Synthetic Data Scales the Efficiency of LLM Math Reasoning by Eight-Foldcited by 99
  7. 2025Flow Matching Policy Gradientscited by 99
  8. 2023CoMPosT: Characterizing and Evaluating Caricature in LLM Simulationscited by 97
  9. 2024How to Train Data-Efficient LLMscited by 93
  10. 2023Identifying and Mitigating the Security Risks of Generative AIcited by 91
  11. 2024A Survey on Self-Evolution of Large Language Modelscited by 91
  12. 2024Understanding Hallucinations in Diffusion Models through Mode Interpolationcited by 91
  13. 2024RLHF Deciphered: A Critical Analysis of Reinforcement Learning from Human Feedback for LLMscited by 90
  14. 2024AgentInstruct: Toward Generative Teaching with Agentic Flowscited by 89
  15. 2023Watermarks in the Sand: Impossibility of Strong Watermarking for Generative Modelscited by 87
  16. 2023When Foundation Model Meets Federated Learning: Motivations, Challenges, and Future Directionscited by 84
  17. 2024Model Collapse Demystified: The Case of Regressioncited by 82
  18. 2023Large Language Models Suffer From Their Own Output: An Analysis of the Self-Consuming Training Loopcited by 79
  19. 2024Functional Benchmarks for Robust Evaluation of Reasoning Performance, and the Reasoning Gapcited by 79
  20. 2024How Bad is Training on Synthetic Data? A Statistical Analysis of Language Model Collapsecited by 79
  21. 2024Feedback Loops With Language Models Drive In-Context Reward Hackingcited by 77
  22. 2024A Design Space for Intelligent and Interactive Writing Assistantscited by 76
  23. 2023Nightshade: Prompt-Specific Poisoning Attacks on Text-to-Image Generative Modelscited by 75
  24. 2023Alignment for Honestycited by 74
  25. 2023GPTEval: A Survey on Assessments of ChatGPT and GPT-4cited by 73
  26. 2023TinyGSM: achieving >80% on GSM8k with small language modelscited by 70
  27. 2024Preserving Diversity in Supervised Fine-Tuning of Large Language Modelscited by 70
  28. 2025Can Large Reasoning Models Self-Train?cited by 68
  29. 2024Strong Model Collapsecited by 65
  30. 2024MoRAL: MoE Augmented LoRA for LLMs' Lifelong Learningcited by 63
  31. 2023Contrasting Linguistic Patterns in Human and LLM-Generated News Textcited by 62
  32. 2024Large Language Model Safety: A Holistic Surveycited by 59
  33. 2024Fairness Feedback Loops: Training on Synthetic Data Amplifies Biascited by 55
  34. 2023Towards Understanding the Interplay of Generative Artificial Intelligence and the Internetcited by 54
  35. 2025Satori: Reinforcement Learning with Chain-of-Action-Thought Enhances LLM Reasoning via Autoregressive Searchcited by 53
  36. 2024Raidar: geneRative AI Detection viA Rewritingcited by 52
  37. 2024Collapse or Thrive? Perils and Promises of Synthetic Data in a Self-Generating Worldcited by 52
  38. 2025The Best Instruction-Tuning Data are Those That Fitcited by 52

50 most cited of 356 All 356 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2026Break Step: Recursive Training Resonates with Replayed Sampling Noise
  2. 2026Experimental evidence of progressive ChatGPT models self-convergence
  3. 2024How to Synthesize Text Data without Model Collapse?
  4. 2026Differences in Text Generated by Diffusion and Autoregressive Language Models
  5. 2024Unstable Unlearning: The Hidden Risk of Concept Resurgence in Diffusion Models
  6. 2024Scaling Diffusion Language Models via Adaptation from Autoregressive Models
  7. 2022Latent Diffusion for Language Generation
  8. 2026Preventing Model Collapse: A Fisher-Rao Perspective on the Dynamics of Training with Synthetic Data