A Markovian View of Iterative-Feedback Loops in Image Generative Models: Neural Resonance and Model Collapse
On the map: Synthetic data and self-improvement for LLMs → Model collapse from synthetic data
Abstract (arXiv)
AI training datasets will inevitably contain AI-generated examples, leading to ``feedback'' in which the output of one model impacts the training of another. It is known that such iterative feedback can lead to model collapse, yet the mechanisms underlying this degeneration remain poorly understood. Here we show that a broad class of feedback processes converges to a low-dimensional invariant structure in latent space, a phenomenon we call neural resonance. By modeling iterative feedback as a Markov Chain, we show that two conditions are needed for this resonance to occur: ergodicity of the feedback process and directional contraction of the latent representation. By studying diffusion models on MNIST and ImageNet, as well as CycleGAN and an audio feedback experiment, we map how local and global manifold geometry evolve, and we introduce an eight-pattern taxonomy of collapse behaviors. Neural resonance provides a unified explanation for long-term degenerate behavior in generative models and provides practical diagnostics for identifying, characterizing, and eventually mitigating collapse.
Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Model collapse from synthetic data, one of 13 directions in Synthetic data and self-improvement for LLMs. Builds on 19 of its 34 references that are in the map. First posted on arXiv 22 Feb 2026.
Builds on
19 of 34 references.
Closest ancestors · 4
Same direction, newest first
- 2025A Closer Look at Model Collapse: From a Generalization-to-Memorization Perspective20 Sep
- 2024Model Collapse in the Self-Consuming Chain of Diffusion Finetuning: A Novel Perspective from Quantitative Trait Modeling4 Jul
- 2024Is Model Collapse Inevitable? Breaking the Curse of Recursion by Accumulating Real and Synthetic Data1 Apr
- 2024A Tale of Tails: Model Collapse as a Change of Scaling Laws10 Feb
Other references · 15
- 2020Denoising Diffusion Probabilistic Modelscited by 20,851
- 2020Score-Based Generative Modeling through Stochastic Differential Equationscited by 8,964
- 2014ImageNet Large Scale Visual Recognition Challengecited by 8,441
- 2021Diffusion Models Beat GANs on Image Synthesiscited by 8,185
- 2019Generative Modeling by Estimating Gradients of the Data Distributioncited by 4,067
- 2020wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representationscited by 3,853
- 2023Self-Consuming Generative Models Go MADcited by 239
- 2023On the Stability of Iterative Retraining of Generative Models on their own Datacited by 98
- 2023The Curious Decline of Linguistic Diversity: Training Language Models on Synthetic Textcited by 97
- 2023Large Language Models Suffer From Their Own Output: An Analysis of the Self-Consuming Training Loopcited by 75
- 2022Data Feedback Loops: Model-driven Amplification of Dataset Biasescited by 59
- 2022Will Large-scale Generative Models Corrupt Future Datasets?cited by 59
3 more
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2025A solvable model of learning generative diffusion: theory and insights
- 2025Convergence Dynamics and Stabilization Strategies of Co-Evolving Generative Models
- 2024Losing dimensions: Geometric memorization in generative diffusion
- 2026The Geometry of Phase Transitions in Generative Dynamics via Projection Caustics
- 2025Transition Models: Rethinking the Generative Learning Objective
- 2026When Does Model Collapse Occur in Structured Interactive Learning?
- 2026Recursively Trained Diffusion Models: Limiting Collapse Distribution and Spectral Characterization
- 2026Memorisation, convergence and generalisation in generative models