mlsanityData to 14 Oct 2026

What happens when generative AI models train recursively on each others' outputs?

arXiv 2505.21677 ↗First posted 27 May 2025cs.LG cs.AI cs.CYcited by 6

Not on the map. Why?

Abstract (arXiv)

The internet serves as a common source of training data for generative AI (genAI) models but is increasingly populated with AI-generated content. This duality raises the possibility that future genAI models may be trained on other models' generated outputs. Prior work has studied consequences of models training on their own generated outputs, but limited work has considered what happens if models ingest content produced by other models. Given society's increasing dependence on genAI tools, understanding such data-mediated model interactions is critical. This work provides empirical evidence for how data-mediated interactions might unfold in practice, develops a theoretical model for this interactive training process, and experimentally validates the theory. We find that data-mediated interactions can benefit models by exposing them to novel concepts perhaps missed in original training data, but also can homogenize their performance on shared tasks.

Key facts. Cited by 6 papers in the map (data through 14 Oct 2026). Builds on 38 of its 61 references that are in the map. First posted on arXiv 27 May 2025.

Builds on

38 of 61 references.

References · 38

  1. 2020Language Models are Few-Shot Learnerscited by 32,419
  2. 2023GPT-4 Technical Reportcited by 18,396
  3. 2024The Llama 3 Herd of Modelscited by 14,715
  4. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 14,132
  5. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 13,028
  6. 2021Training Verifiers to Solve Math Word Problemscited by 10,109
  7. 2020Scaling Laws for Neural Language Modelscited by 6,586
  8. 2022PaLM: Scaling Language Modeling with Pathwayscited by 5,326
  9. 2024Gemini 1.5: Unlocking multimodal understanding across millions of tokens of contextcited by 3,224
  10. 2022Training Compute-Optimal Large Language Modelscited by 3,019
  11. 2024Phi-3 Technical Report: A Highly Capable Language Model Locally on Your Phonecited by 1,917
  12. 2022LaMDA: Language Models for Dialog Applicationscited by 1,220
26 more
  1. 2019A Comprehensive Survey on Transfer Learningcited by 1,160
  2. 2021GLaM: Efficient Scaling of Language Models with Mixture-of-Expertscited by 897
  3. 2017Crowdsourcing Multiple Choice Science Questionscited by 765
  4. 2023Textbooks Are All You Need II: phi-1.5 technical reportcited by 565
  5. 2023Can AI-Generated Text be Reliably Detected?cited by 386
  6. 2023Self-Consuming Generative Models Go MADcited by 250
  7. 2024Is Model Collapse Inevitable? Breaking the Curse of Recursion by Accumulating Real and Synthetic Datacited by 151
  8. 2025Which Economic Tasks are Performed with AI? Evidence from Millions of Claude Conversationscited by 96
  9. 2023Watermarks in the Sand: Impossibility of Strong Watermarking for Generative Modelscited by 87
  10. 2024Model Collapse Demystified: The Case of Regressioncited by 82
  11. 2022Will Large-scale Generative Models Corrupt Future Datasets?cited by 59
  12. 2023Towards Understanding the Interplay of Generative Artificial Intelligence and the Internetcited by 54
  13. 2024Collapse or Thrive? Perils and Promises of Synthetic Data in a Self-Generating Worldcited by 52
  14. 2024Jamba-1.5: Hybrid Transformer-Mamba Models at Scalecited by 50
  15. 2025We're Different, We're the Same: Creative Homogeneity Across LLMscited by 42
  16. 2024Are We in the AI-Generated Text World Already? Quantifying and Monitoring AIGT on Social Mediacited by 35
  17. 2024AI and the Problem of Knowledge Collapsecited by 34
  18. 2025Position: Model Collapse Does Not Mean What You Thinkcited by 33
  19. 2024Universality of the π2/6 Pathway in Avoiding Model Collapsecited by 26
  20. 2024Heat Death of Generative Models in Closed-Loop Learningcited by 22
  21. 2024Regurgitative Training: The Value of Real Data in Training Large Language Modelscited by 15
  22. 2025Machine-generated text detection prevents language model collapsecited by 14
  23. 2025When Models Don't Collapse: On the Consistency of Iterative MLEcited by 13
  24. 2024Examining the Prevalence and Dynamics of AI-Generated Media in Art Subredditscited by 8
  25. 2025Enhancing Domain-Specific Encoder Models with LLM-Generated Data: How to Leverage Ontologies, and How to Do Without Themcited by 4
  26. 2025Generative Distribution Prediction: A Unified Approach to Multimodal Learningcited by 1

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 6.

6 papers.

  1. 2026Quantifying Error Propagation and Model Collapse in Diffusion Modelscited by 4
  2. 2026When Does Model Collapse Occur in Structured Interactive Learning?cited by 2
  3. 2026Are LLMs becoming similarly creative? Evidence from three years of modelscited by 2
  4. 2026Minimax bounds for watermarked and masked recursive discrete distribution estimationcited by 0
  5. 2026The Oligarch Barely Steers Model Collapse in Multi-Model Ecosystemscited by 0
  6. 2026Stability and Diversity of Networked Self-Consuming Generative Ecosystemscited by 0