mlsanityData to 13 Oct 2026

Combining Generative Artificial Intelligence (AI) and the Internet: Heading towards Evolution or Degradation?

arXiv 2303.01255 ↗First posted 17 Feb 2023cs.CVcited by 41

Not on the map. Why?

Abstract (arXiv)

In the span of a few months, generative Artificial Intelligence (AI) tools that can generate realistic images or text have taken the Internet by storm, making them one of the technologies with fastest adoption ever. Some of these generative AI tools such as DALL-E, MidJourney, or ChatGPT have gained wide public notoriety. Interestingly, these tools are possible because of the massive amount of data (text and images) available on the Internet. The tools are trained on massive data sets that are scraped from Internet sites. And now, these generative AI tools are creating massive amounts of new data that are being fed into the Internet. Therefore, future versions of generative AI tools will be trained with Internet data that is a mix of original and AI-generated data. As time goes on, a mixture of original data and data generated by different versions of AI tools will populate the Internet. This raises a few intriguing questions: how will future versions of generative AI tools behave when trained on a mixture of real and AI generated data? Will they evolve with the new data sets or degenerate? Will evolution introduce biases in subsequent generations of generative AI tools? In this document, we explore these questions and report some very initial simulation results using a simple image-generation AI tool. These results suggest that the quality of the generated images degrades as more AI-generated data is used for training thus suggesting that generative AI may degenerate. Although these results are preliminary and cannot be generalised without further study, they serve to illustrate the potential issues of the interaction between generative AI and the Internet.

Key facts. Cited by 41 papers in the map (data through 13 Oct 2026). Builds on 8 of its 11 references that are in the map. First posted on arXiv 17 Feb 2023.

Builds on

8 of 11 references.

References · 8

  1. 2022Hierarchical Text-Conditional Image Generation with CLIP Latentscited by 6,564
  2. 2022LAION-5B: An open large-scale dataset for training next generation image-text modelscited by 4,141
  3. 2021LAION-400M: Open Dataset of CLIP-Filtered 400 Million Image-Text Pairscited by 1,548
  4. 2021CogView: Mastering Text-to-Image Generation via Transformerscited by 692
  5. 2023ChatGPT is not all you need. A State of the Art Review of large Generative AI modelscited by 86
  6. 2022Will Large-scale Generative Models Corrupt Future Datasets?cited by 59
  7. 2022Generated Faces in the Wild: Quantitative Comparison of Stable Diffusion, Midjourney and DALL-E 2cited by 55
  8. 2023Is AI Art Another Industrial Revolution in the Making?cited by 5

Built on by

Cited by year (2026 to 13 Oct)

Citing papers per year: 2023: 6; 2024: 18; 2025: 12; 2026: 5.

41 papers.

Sort:

By area:

  1. 2023Self-Consuming Generative Models Go MADcited by 246
  2. 2024Is Model Collapse Inevitable? Breaking the Curse of Recursion by Accumulating Real and Synthetic Datacited by 148
  3. 2024A Tale of Tails: Model Collapse as a Change of Scaling Lawscited by 128
  4. 2023On the Stability of Iterative Retraining of Generative Models on their own Datacited by 98
  5. 2024Understanding Hallucinations in Diffusion Models through Mode Interpolationcited by 88
  6. 2024Model Collapse Demystified: The Case of Regressioncited by 81
  7. 2024Strong Model Collapsecited by 64
  8. 2023Towards Understanding the Interplay of Generative Artificial Intelligence and the Internetcited by 54
  9. 2025The Best Instruction-Tuning Data are Those That Fitcited by 50
  10. 2024Collapse or Thrive? Perils and Promises of Synthetic Data in a Self-Generating Worldcited by 49
  11. 2024Beyond Model Collapse: Scaling Up with Synthesized Data Requires Verificationcited by 45
  12. 2024Self-Consuming Generative Models with Curated Data Provably Optimize Human Preferencescited by 39
29 more
  1. 2025Self-Improving Transformers Overcome Easy-to-Hard and Length Generalization Challengescited by 30
  2. 2024Towards Theoretical Understandings of Self-Consuming Generative Modelscited by 28
  3. 2023Nepotistically Trained Generative-AI Models Collapsecited by 27
  4. 2024Universality of the π2/6 Pathway in Avoiding Model Collapsecited by 25
  5. 2024Rate of Model Collapse in Recursive Trainingcited by 24
  6. 2024Heat Death of Generative Models in Closed-Loop Learningcited by 22
  7. 2024Mind the Gap: Examining the Self-Improvement Capabilities of Large Language Modelscited by 18
  8. 2024Model Collapse in the Self-Consuming Chain of Diffusion Finetuning: A Novel Perspective from Quantitative Trait Modelingcited by 11
  9. 2024Analyzing and Mitigating Model Collapse in Rectified Flow Modelscited by 10
  10. 2025Preventing Model Collapse Under Overparametrization: Optimal Mixing Ratios for Interpolation Learning and Ridge Regressioncited by 9
  11. 2025Why Less is More (Sometimes): A Theory of Data Curationcited by 8
  12. 2023Hazards from Increasingly Accessible Fine-Tuning of Downloadable Foundation Modelscited by 7
  13. 2023How many words does ChatGPT know? The answer is ChatWordscited by 6
  14. 2025LLM Web Dynamics: Tracing Model Collapse in a Network of LLMscited by 6
  15. 2024When AI Eats Itself: On the Caveats of AI Autophagycited by 4
  16. 2025The Imitation Game: Turing Machine Imitator is Length Generalizable Reasonercited by 3
  17. 2025Coherence Mechanisms for Provable Self-Improvementcited by 3
  18. 2025Convergence and Stability Analysis of Self-Consuming Generative Models with Heterogeneous Human Curationcited by 3
  19. 2024Mitigating Text Toxicity with Counterfactual Generationcited by 2
  20. 2024Learning by Surprise: Adaptive Mitigation of Model Collapse in Large Language Modelscited by 2
  21. 2025Convergence Dynamics and Stabilization Strategies of Co-Evolving Generative Modelscited by 2
  22. 2025Recursive Training Loops in LLMs: How training data properties modulate distribution shift in generated data?cited by 2
  23. 2025Review of Hallucination Understanding in Large Language and Vision Modelscited by 2
  24. 2025AI-generated Text Detection with a GLTR-based Approachcited by 1
  25. 2026From Collapse to Improvement: Statistical Perspectives on the Evolutionary Dynamics of Iterative Training on Contaminated Sourcescited by 1
  26. 2026When Does Model Collapse Occur in Structured Interactive Learning?cited by 1
  27. 2026Can Generative Artificial Intelligence Survive Data Contamination? Theoretical Guarantees under Contaminated Recursive Trainingcited by 0
  28. 2026The Economics of Model Collapse: Equilibrium, Welfare, and Optimal Provenance Subsidies in Synthetic Data Marketscited by 0
  29. 2026Optimal Self-Distillation for Rectified Flow via Linear Probingcited by 0