# Convergence Dynamics and Stabilization Strategies of Co-Evolving Generative Models

arXiv: https://arxiv.org/abs/2503.08117 · HTML: https://mlsanity.com/paper/2503.08117/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Placed in Model collapse from synthetic data, one of 13 directions in Synthetic data and self-improvement for LLMs.
- Builds on 21 of its 31 references that are in the map.
- First posted on arXiv 11 Mar 2025.

## Placed in

- Synthetic data and self-improvement for LLMs / [Model collapse from synthetic data](https://mlsanity.com/synth/model-collapse-from-synthetic-data/)

## Builds on (most cited first, 21 of 21)

- [High-Resolution Image Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2112.10752/) (2021)
- [Self-Consuming Generative Models Go MAD](https://mlsanity.com/paper/2307.01850/) (2023)
- [Flow Matching in Latent Space](https://mlsanity.com/paper/2307.08698/) (2023)
- [Is Model Collapse Inevitable? Breaking the Curse of Recursion by Accumulating Real and Synthetic Data](https://mlsanity.com/paper/2404.01413/) (2024)
- [A Tale of Tails: Model Collapse as a Change of Scaling Laws](https://mlsanity.com/paper/2402.07043/) (2024)
- [On the Stability of Iterative Retraining of Generative Models on their own Data](https://mlsanity.com/paper/2310.00429/) (2023)
- [The Curious Decline of Linguistic Diversity: Training Language Models on Synthetic Text](https://mlsanity.com/paper/2311.09807/) (2023)
- [Model Collapse Demystified: The Case of Regression](https://mlsanity.com/paper/2402.07712/) (2024)
- [Feedback Loops With Language Models Drive In-Context Reward Hacking](https://mlsanity.com/paper/2402.06627/) (2024)
- [Data Feedback Loops: Model-driven Amplification of Dataset Biases](https://mlsanity.com/paper/2209.03942/) (2022)
- [Fairness Feedback Loops: Training on Synthetic Data Amplifies Bias](https://mlsanity.com/paper/2403.07857/) (2024)
- [Towards Understanding the Interplay of Generative Artificial Intelligence and the Internet](https://mlsanity.com/paper/2306.06130/) (2023)
- [Combining Generative Artificial Intelligence (AI) and the Internet: Heading towards Evolution or Degradation?](https://mlsanity.com/paper/2303.01255/) (2023)
- [Self-Consuming Generative Models with Curated Data Provably Optimize Human Preferences](https://mlsanity.com/paper/2407.09499/) (2024)
- [Self-Correcting Self-Consuming Loops for Generative Model Training](https://mlsanity.com/paper/2402.07087/) (2024)
- [AI and the Problem of Knowledge Collapse](https://mlsanity.com/paper/2404.03502/) (2024)
- [Nepotistically Trained Generative-AI Models Collapse](https://mlsanity.com/paper/2311.12202/) (2023)
- [ChatGPT "contamination": estimating the prevalence of LLMs in the scholarly literature](https://mlsanity.com/paper/2403.16887/) (2024)
- [Rate of Model Collapse in Recursive Training](https://mlsanity.com/paper/2412.17646/) (2024)
- [Heat Death of Generative Models in Closed-Loop Learning](https://mlsanity.com/paper/2404.02325/) (2024)
- [Analyzing Recursiveness in Multimodal Generative Artificial Intelligence: Stability or Divergence?](https://mlsanity.com/paper/2409.16297/) (2024)

## Built on by (most cited first, 2 of 2)

- [When and How Human Curation Backfires: Preference Alignment under Multi-Model Self-Consuming Loop](https://mlsanity.com/paper/2605.29267/) (2026)
- [Stability and Diversity of Networked Self-Consuming Generative Ecosystems](https://mlsanity.com/paper/2610.09409/) (2026)

## Abstract (arXiv)

The increasing prevalence of synthetic data in training loops has raised concerns about model collapse, where generative models degrade when trained on their own outputs. While prior work focuses on this self-consuming process, we study an underexplored yet prevalent phenomenon: co-evolving generative models that shape each other's training through iterative feedback. This is common in multimodal AI ecosystems, such as social media platforms, where text models generate captions that guide image models, and the resulting images influence the future adaptation of the text model. We take a first step by analyzing such a system, modeling the text model as a multinomial distribution and the image model as a conditional multi-dimensional Gaussian distribution. Our analysis uncovers three key results. First, when one model remains fixed, the other collapses: a frozen image model causes the text model to lose diversity, while a frozen text model leads to an exponential contraction of image diversity, though fidelity remains bounded. Second, in fully interactive systems, mutual reinforcement accelerates collapse, with image contraction amplifying text homogenization and vice versa, leading to a Matthew effect where dominant texts sustain higher image diversity while rarer texts collapse faster. Third, we analyze stabilization strategies implicitly introduced by real-world external influences. Random corpus injections for text models and user-content injections for image models prevent collapse while preserving both diversity and fidelity. Our theoretical findings are further validated through experiments.