mlsanityData to 7 Oct 2026

Can Generative Artificial Intelligence Survive Data Contamination? Theoretical Guarantees under Contaminated Recursive Training

arXiv 2602.16065 ↗First posted 17 Feb 2026cs.LG cs.AI math.STcited by 0

Not on the map. Why?

Abstract (arXiv)

As artificial intelligence (AI)-generated content proliferates, models are increasingly trained on their own outputs, risking progressive degradation or collapse. In this article, we provide the first positive, rigorous theoretical results, to the best of our knowledge, showing that under model-agnostic mild conditions, the model converges to the true data-generating distribution. The convergence rate is the minimum of the model's intrinsic rate and the fraction of real data at each training iteration, revealing a phase transition between data-limited and model-limited regimes. We further show that, for biased real data, correcting the bias prevents the persistence and amplification of early bias over training iteration. Extensive experiments across simulations, real images and texts validate our theoretical framework, establishing quantitative conditions for long-term AI stability in contaminated environments.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 44 of its 63 references that are in the map. First posted on arXiv 17 Feb 2026.

Builds on

44 of 63 references.

References · 44

  1. 2015Distilling the Knowledge in a Neural Networkcited by 10,076
  2. 2019DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lightercited by 4,243
  3. 2019A Survey on Bias and Fairness in Machine Learningcited by 1,911
  4. 2017Domain Randomization for Transferring Deep Neural Networks from Simulation to the Real Worldcited by 1,401
  5. 2019Self-training with Noisy Student improves ImageNet classificationcited by 1,304
  6. 2018Which Training Methods for GANs do actually Converge?cited by 672
  7. 2019The Woman Worked as a Babysitter: On Biases in Language Generationcited by 540
  8. 2018Born Again Neural Networkscited by 539
  9. 2019Pseudo-Labeling and Confirmation Bias in Deep Semi-Supervised Learningcited by 507
  10. 2019Dreaming to Distill: Data-free Knowledge Transfer via DeepInversioncited by 419
  11. 2016Semi-supervised Knowledge Transfer for Deep Learning from Private Training Datacited by 347
  12. 2023The Curse of Recursion: Training on Generated Data Makes Models Forgetcited by 343
32 more
  1. 2022Convergence of denoising diffusion models under the manifold hypothesiscited by 237
  2. 2018Differentially Private Generative Adversarial Networkcited by 235
  3. 2021SynthSeg: Segmentation of brain MRI scans of any contrast and resolution without retrainingcited by 235
  4. 2016CAD2RL: Real Single-Image Flight without a Single Real Imagecited by 227
  5. 2023Score Approximation, Estimation and Distribution Recovery of Diffusion Models on Low-Dimensional Datacited by 171
  6. 2024Monitoring AI-Modified Content at Scale: A Case Study on the Impact of ChatGPT on AI Conference Peer Reviewscited by 160
  7. 2023Diffusion Models are Minimax Optimal Distribution Estimatorscited by 159
  8. 2024Is Model Collapse Inevitable? Breaking the Curse of Recursion by Accumulating Real and Synthetic Datacited by 143
  9. 2017Data-Free Knowledge Distillation for Deep Neural Networkscited by 136
  10. 2024A Tale of Tails: Model Collapse as a Change of Scaling Lawscited by 127
  11. 2018GAN Augmentation: Augmenting Training Data using Generative Adversarial Networkscited by 107
  12. 2023On the Stability of Iterative Retraining of Generative Models on their own Datacited by 98
  13. 2024Model Collapse Demystified: The Case of Regressioncited by 80
  14. 2023Large Language Models Suffer From Their Own Output: An Analysis of the Self-Consuming Training Loopcited by 75
  15. 2024Strong Model Collapsecited by 62
  16. 2022Will Large-scale Generative Models Corrupt Future Datasets?cited by 59
  17. 2024Beyond Model Collapse: Scaling Up with Synthesized Data Requires Verificationcited by 43
  18. 2023Combining Generative Artificial Intelligence (AI) and the Internet: Heading towards Evolution or Degradation?cited by 41
  19. 2022A Comprehensive Empirical Study of Bias Mitigation Methods for Machine Learning Classifierscited by 30
  20. 2024Unlocking Tokens as Data Points for Generalization Bounds on Larger Language Modelscited by 25
  21. 2024Universality of the π2/6 Pathway in Avoiding Model Collapsecited by 24
  22. 2019Nonparametric Density Estimation & Convergence Rates for GANs under Besov IPM Lossescited by 22
  23. 2024The Rise of AI-Generated Content in Wikipediacited by 22
  24. 2024Rate of Model Collapse in Recursive Trainingcited by 22
  25. 2022On PAC-Bayesian reconstruction guarantees for VAEscited by 20
  26. 2024Bias in Generative AIcited by 19
  27. 2023Statistical Guarantees for Variational Autoencoders using PAC-Bayesian Theorycited by 16
  28. 2024Regurgitative Training: The Value of Real Data in Training Large Language Modelscited by 15
  29. 2024A Little Help Goes a Long Way: Efficient LLM Training by Leveraging Small LMscited by 15
  30. 2017How Well Can Generative Adversarial Networks Learn Densities: A Nonparametric Viewcited by 14
  31. 2021Rates of convergence for density estimation with generative adversarial networkscited by 13
  32. 2025When Models Don't Collapse: On the Consistency of Iterative MLEcited by 11

Built on by

Not cited yet.