# Combining Generative Artificial Intelligence (AI) and the Internet: Heading towards Evolution or Degradation? arXiv: https://arxiv.org/abs/2303.01255 · HTML: https://mlsanity.com/paper/2303.01255/ ## Key facts - Cited by 41 papers in the map (data through 14 Oct 2026). - Builds on 8 of its 11 references that are in the map. - First posted on arXiv 17 Feb 2023. ## Builds on (most cited first, 8 of 8) - [Hierarchical Text-Conditional Image Generation with CLIP Latents](https://mlsanity.com/paper/2204.06125/) (2022) - [LAION-5B: An open large-scale dataset for training next generation image-text models](https://mlsanity.com/paper/2210.08402/) (2022) - [LAION-400M: Open Dataset of CLIP-Filtered 400 Million Image-Text Pairs](https://mlsanity.com/paper/2111.02114/) (2021) - [CogView: Mastering Text-to-Image Generation via Transformers](https://mlsanity.com/paper/2105.13290/) (2021) - [ChatGPT is not all you need. A State of the Art Review of large Generative AI models](https://mlsanity.com/paper/2301.04655/) (2023) - [Will Large-scale Generative Models Corrupt Future Datasets?](https://mlsanity.com/paper/2211.08095/) (2022) - [Generated Faces in the Wild: Quantitative Comparison of Stable Diffusion, Midjourney and DALL-E 2](https://mlsanity.com/paper/2210.00586/) (2022) - [Is AI Art Another Industrial Revolution in the Making?](https://mlsanity.com/paper/2301.05133/) (2023) ## Built on by (most cited first, 30 of 41) - [Self-Consuming Generative Models Go MAD](https://mlsanity.com/paper/2307.01850/) (2023) - [Is Model Collapse Inevitable? Breaking the Curse of Recursion by Accumulating Real and Synthetic Data](https://mlsanity.com/paper/2404.01413/) (2024) - [A Tale of Tails: Model Collapse as a Change of Scaling Laws](https://mlsanity.com/paper/2402.07043/) (2024) - [On the Stability of Iterative Retraining of Generative Models on their own Data](https://mlsanity.com/paper/2310.00429/) (2023) - [Understanding Hallucinations in Diffusion Models through Mode Interpolation](https://mlsanity.com/paper/2406.09358/) (2024) - [Model Collapse Demystified: The Case of Regression](https://mlsanity.com/paper/2402.07712/) (2024) - [Strong Model Collapse](https://mlsanity.com/paper/2410.04840/) (2024) - [Towards Understanding the Interplay of Generative Artificial Intelligence and the Internet](https://mlsanity.com/paper/2306.06130/) (2023) - [Collapse or Thrive? Perils and Promises of Synthetic Data in a Self-Generating World](https://mlsanity.com/paper/2410.16713/) (2024) - [The Best Instruction-Tuning Data are Those That Fit](https://mlsanity.com/paper/2502.04194/) (2025) - [Beyond Model Collapse: Scaling Up with Synthesized Data Requires Verification](https://mlsanity.com/paper/2406.07515/) (2024) - [Self-Consuming Generative Models with Curated Data Provably Optimize Human Preferences](https://mlsanity.com/paper/2407.09499/) (2024) - [Self-Improving Transformers Overcome Easy-to-Hard and Length Generalization Challenges](https://mlsanity.com/paper/2502.01612/) (2025) - [Towards Theoretical Understandings of Self-Consuming Generative Models](https://mlsanity.com/paper/2402.11778/) (2024) - [Nepotistically Trained Generative-AI Models Collapse](https://mlsanity.com/paper/2311.12202/) (2023) - [Universality of the π2/6 Pathway in Avoiding Model Collapse](https://mlsanity.com/paper/2410.22812/) (2024) - [Rate of Model Collapse in Recursive Training](https://mlsanity.com/paper/2412.17646/) (2024) - [Heat Death of Generative Models in Closed-Loop Learning](https://mlsanity.com/paper/2404.02325/) (2024) - [Mind the Gap: Examining the Self-Improvement Capabilities of Large Language Models](https://mlsanity.com/paper/2412.02674/) (2024) - [Model Collapse in the Self-Consuming Chain of Diffusion Finetuning: A Novel Perspective from Quantitative Trait Modeling](https://mlsanity.com/paper/2407.17493/) (2024) - [Analyzing and Mitigating Model Collapse in Rectified Flow Models](https://mlsanity.com/paper/2412.08175/) (2024) - [Preventing Model Collapse Under Overparametrization: Optimal Mixing Ratios for Interpolation Learning and Ridge Regression](https://mlsanity.com/paper/2509.22341/) (2025) - [Why Less is More (Sometimes): A Theory of Data Curation](https://mlsanity.com/paper/2511.03492/) (2025) - [Hazards from Increasingly Accessible Fine-Tuning of Downloadable Foundation Models](https://mlsanity.com/paper/2312.14751/) (2023) - [How many words does ChatGPT know? The answer is ChatWords](https://mlsanity.com/paper/2309.16777/) (2023) - [LLM Web Dynamics: Tracing Model Collapse in a Network of LLMs](https://mlsanity.com/paper/2506.15690/) (2025) - [When AI Eats Itself: On the Caveats of AI Autophagy](https://mlsanity.com/paper/2405.09597/) (2024) - [The Imitation Game: Turing Machine Imitator is Length Generalizable Reasoner](https://mlsanity.com/paper/2507.13332/) (2025) - [Coherence Mechanisms for Provable Self-Improvement](https://mlsanity.com/paper/2511.08440/) (2025) - [Convergence and Stability Analysis of Self-Consuming Generative Models with Heterogeneous Human Curation](https://mlsanity.com/paper/2511.09002/) (2025) ## Abstract (arXiv) In the span of a few months, generative Artificial Intelligence (AI) tools that can generate realistic images or text have taken the Internet by storm, making them one of the technologies with fastest adoption ever. Some of these generative AI tools such as DALL-E, MidJourney, or ChatGPT have gained wide public notoriety. Interestingly, these tools are possible because of the massive amount of data (text and images) available on the Internet. The tools are trained on massive data sets that are scraped from Internet sites. And now, these generative AI tools are creating massive amounts of new data that are being fed into the Internet. Therefore, future versions of generative AI tools will be trained with Internet data that is a mix of original and AI-generated data. As time goes on, a mixture of original data and data generated by different versions of AI tools will populate the Internet. This raises a few intriguing questions: how will future versions of generative AI tools behave when trained on a mixture of real and AI generated data? Will they evolve with the new data sets or degenerate? Will evolution introduce biases in subsequent generations of generative AI tools? In this document, we explore these questions and report some very initial simulation results using a simple image-generation AI tool. These results suggest that the quality of the generated images degrades as more AI-generated data is used for training thus suggesting that generative AI may degenerate. Although these results are preliminary and cannot be generalised without further study, they serve to illustrate the potential issues of the interaction between generative AI and the Internet.