Can Generative Artificial Intelligence Survive Data Contamination? Theoretical Guarantees under Contaminated Recursive Training
Not on the map. Why?
Abstract (arXiv)
As artificial intelligence (AI)-generated content proliferates, models are increasingly trained on their own outputs, risking progressive degradation or collapse. In this article, we provide the first positive, rigorous theoretical results, to the best of our knowledge, showing that under model-agnostic mild conditions, the model converges to the true data-generating distribution. The convergence rate is the minimum of the model's intrinsic rate and the fraction of real data at each training iteration, revealing a phase transition between data-limited and model-limited regimes. We further show that, for biased real data, correcting the bias prevents the persistence and amplification of early bias over training iteration. Extensive experiments across simulations, real images and texts validate our theoretical framework, establishing quantitative conditions for long-term AI stability in contaminated environments.
Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 44 of its 63 references that are in the map. First posted on arXiv 17 Feb 2026.
Builds on
44 of 63 references.
References · 44
- 2015Distilling the Knowledge in a Neural Networkcited by 10,076
- 2019DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lightercited by 4,243
- 2019A Survey on Bias and Fairness in Machine Learningcited by 1,911
- 2017Domain Randomization for Transferring Deep Neural Networks from Simulation to the Real Worldcited by 1,401
- 2019Self-training with Noisy Student improves ImageNet classificationcited by 1,304
- 2018Which Training Methods for GANs do actually Converge?cited by 672
- 2019The Woman Worked as a Babysitter: On Biases in Language Generationcited by 540
- 2018Born Again Neural Networkscited by 539
- 2019Pseudo-Labeling and Confirmation Bias in Deep Semi-Supervised Learningcited by 507
- 2019Dreaming to Distill: Data-free Knowledge Transfer via DeepInversioncited by 419
- 2016Semi-supervised Knowledge Transfer for Deep Learning from Private Training Datacited by 347
- 2023The Curse of Recursion: Training on Generated Data Makes Models Forgetcited by 343
32 more
- 2022Convergence of denoising diffusion models under the manifold hypothesiscited by 237
- 2018Differentially Private Generative Adversarial Networkcited by 235
- 2021SynthSeg: Segmentation of brain MRI scans of any contrast and resolution without retrainingcited by 235
- 2016CAD2RL: Real Single-Image Flight without a Single Real Imagecited by 227
- 2023Score Approximation, Estimation and Distribution Recovery of Diffusion Models on Low-Dimensional Datacited by 171
- 2024Monitoring AI-Modified Content at Scale: A Case Study on the Impact of ChatGPT on AI Conference Peer Reviewscited by 160
- 2023Diffusion Models are Minimax Optimal Distribution Estimatorscited by 159
- 2024Is Model Collapse Inevitable? Breaking the Curse of Recursion by Accumulating Real and Synthetic Datacited by 143
- 2017Data-Free Knowledge Distillation for Deep Neural Networkscited by 136
- 2024A Tale of Tails: Model Collapse as a Change of Scaling Lawscited by 127
- 2018GAN Augmentation: Augmenting Training Data using Generative Adversarial Networkscited by 107
- 2023On the Stability of Iterative Retraining of Generative Models on their own Datacited by 98
- 2024Model Collapse Demystified: The Case of Regressioncited by 80
- 2023Large Language Models Suffer From Their Own Output: An Analysis of the Self-Consuming Training Loopcited by 75
- 2024Strong Model Collapsecited by 62
- 2022Will Large-scale Generative Models Corrupt Future Datasets?cited by 59
- 2024Beyond Model Collapse: Scaling Up with Synthesized Data Requires Verificationcited by 43
- 2023Combining Generative Artificial Intelligence (AI) and the Internet: Heading towards Evolution or Degradation?cited by 41
- 2022A Comprehensive Empirical Study of Bias Mitigation Methods for Machine Learning Classifierscited by 30
- 2024Unlocking Tokens as Data Points for Generalization Bounds on Larger Language Modelscited by 25
- 2024Universality of the π2/6 Pathway in Avoiding Model Collapsecited by 24
- 2019Nonparametric Density Estimation & Convergence Rates for GANs under Besov IPM Lossescited by 22
- 2024The Rise of AI-Generated Content in Wikipediacited by 22
- 2024Rate of Model Collapse in Recursive Trainingcited by 22
- 2022On PAC-Bayesian reconstruction guarantees for VAEscited by 20
- 2024Bias in Generative AIcited by 19
- 2023Statistical Guarantees for Variational Autoencoders using PAC-Bayesian Theorycited by 16
- 2024Regurgitative Training: The Value of Real Data in Training Large Language Modelscited by 15
- 2024A Little Help Goes a Long Way: Efficient LLM Training by Leveraging Small LMscited by 15
- 2017How Well Can Generative Adversarial Networks Learn Densities: A Nonparametric Viewcited by 14
- 2021Rates of convergence for density estimation with generative adversarial networkscited by 13
- 2025When Models Don't Collapse: On the Consistency of Iterative MLEcited by 11
Built on by
Not cited yet.