# Stabilizing Self-Consuming Diffusion Models with Latent Space Filtering

arXiv: https://arxiv.org/abs/2511.12742 · HTML: https://mlsanity.com/paper/2511.12742/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Placed in Diffusion foundations and sampling, one of 14 directions in Diffusion models.
- Builds on 28 of its 37 references that are in the map.
- First posted on arXiv 16 Nov 2025.

## Placed in

- Diffusion models / [Diffusion foundations and sampling](https://mlsanity.com/diffusion/diffusion-foundations-and-sampling/)
- Synthetic data and self-improvement for LLMs / [Model collapse from synthetic data](https://mlsanity.com/synth/model-collapse-from-synthetic-data/)

## Builds on (most cited first, 28 of 28)

- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [Diffusion Models Beat GANs on Image Synthesis](https://mlsanity.com/paper/2105.05233/) (2021)
- [Generative Modeling by Estimating Gradients of the Data Distribution](https://mlsanity.com/paper/1907.05600/) (2019)
- [Deep Learning Face Attributes in the Wild](https://mlsanity.com/paper/1411.7766/) (2014)
- [Improved Precision and Recall Metric for Assessing Generative Models](https://mlsanity.com/paper/1904.06991/) (2019)
- [The Curse of Recursion: Training on Generated Data Makes Models Forget](https://mlsanity.com/paper/2305.17493/) (2023)
- [Diffusion Models already have a Semantic Latent Space](https://mlsanity.com/paper/2210.10960/) (2022)
- [Self-Consuming Generative Models Go MAD](https://mlsanity.com/paper/2307.01850/) (2023)
- [Intriguing properties of synthetic images: from generative adversarial networks to diffusion models](https://mlsanity.com/paper/2304.06408/) (2023)
- [A Note on Shumailov et al. (2024): `AI Models Collapse When Trained on Recursively Generated Data'](https://mlsanity.com/paper/2410.12954/) (2024)
- [On the Stability of Iterative Retraining of Generative Models on their own Data](https://mlsanity.com/paper/2310.00429/) (2023)
- [Seeing is not always believing: Benchmarking Human and Model Perception of AI-Generated Images](https://mlsanity.com/paper/2304.13023/) (2023)
- [Data Feedback Loops: Model-driven Amplification of Dataset Biases](https://mlsanity.com/paper/2209.03942/) (2022)
- [Breaking the Curse of Dimensionality: Diffusion Models Efficiently Learn Low-Dimensional Distributions](https://mlsanity.com/paper/2409.02426/) (2024)
- [Collapse or Thrive? Perils and Promises of Synthetic Data in a Self-Generating World](https://mlsanity.com/paper/2410.16713/) (2024)
- [Scalable Sparse Subspace Clustering by Orthogonal Matching Pursuit](https://mlsanity.com/paper/1507.01238/) (2015)
- [Beyond Model Collapse: Scaling Up with Synthesized Data Requires Verification](https://mlsanity.com/paper/2406.07515/) (2024)
- [OL'E: Orthogonal Low-rank Embedding, A Plug and Play Geometric Loss for Deep Learning](https://mlsanity.com/paper/1712.01727/) (2017)
- [Self-Improving Diffusion Models with Synthetic Data](https://mlsanity.com/paper/2408.16333/) (2024)
- [Towards Theoretical Understandings of Self-Consuming Generative Models](https://mlsanity.com/paper/2402.11778/) (2024)
- [Synth2: Boosting Visual-Language Models with Synthetic Captions and Image Embeddings](https://mlsanity.com/paper/2403.07750/) (2024)
- [Understanding Representation Dynamics of Diffusion Models via Low-Dimensional Modeling](https://mlsanity.com/paper/2502.05743/) (2025)
- [A Theoretical Perspective: How to Prevent Model Collapse in Self-consuming Training Loops](https://mlsanity.com/paper/2502.18865/) (2025)
- [Superficial Self-Improved Reasoners Benefit from Model Merging](https://mlsanity.com/paper/2503.02103/) (2025)
- [Machine-generated text detection prevents language model collapse](https://mlsanity.com/paper/2502.15654/) (2025)
- [Self-Consuming Generative Models with Adversarially Curated Data](https://mlsanity.com/paper/2505.09768/) (2025)

## Built on by (most cited first, 4 of 4)

- [Observations and Remedies for Large Language Model Bias in Self-Consuming Performative Loop](https://mlsanity.com/paper/2601.05184/) (2026)
- [When and How Human Curation Backfires: Preference Alignment under Multi-Model Self-Consuming Loop](https://mlsanity.com/paper/2605.29267/) (2026)
- [Learning to Discretize: Diffusion-Based Adaptive Mesh with Spectral Guidance](https://mlsanity.com/paper/2607.11974/) (2026)
- [Stability and Diversity of Networked Self-Consuming Generative Ecosystems](https://mlsanity.com/paper/2610.09409/) (2026)

## Abstract (arXiv)

As synthetic data proliferates across the Internet, it is often reused to train successive generations of generative models. This creates a ``self-consuming loop" that can lead to training instability or textitmodel collapse. Common strategies to address the issue -- such as accumulating historical training data or injecting fresh real data -- either increase computational cost or require expensive human annotation. In this paper, we empirically analyze the latent space dynamics of self-consuming diffusion models and observe that the low-dimensional structure of latent representations extracted from synthetic data degrade over generations. Based on this insight, we propose textitLatent Space Filtering (LSF), a novel approach that mitigates model collapse by filtering out less realistic synthetic data from mixed datasets. Theoretically, we present a framework that connects latent space degradation to empirical observations. Experimentally, we show that LSF consistently outperforms existing baselines across multiple real-world datasets, effectively mitigating model collapse without increasing training cost or relying on human annotation.