# Analyzing and Mitigating Model Collapse in Rectified Flow Models arXiv: https://arxiv.org/abs/2412.08175 ยท HTML: https://mlsanity.com/paper/2412.08175/ ## Key facts - Cited by 11 papers in the map (data through 14 Oct 2026). - Placed in Diffusion foundations and sampling, one of 14 directions in Diffusion models. - Builds on 27 of its 114 references that are in the map. - First posted on arXiv 11 Dec 2024. ## Placed in - Diffusion models / [Diffusion foundations and sampling](https://mlsanity.com/diffusion/diffusion-foundations-and-sampling/) - Synthetic data and self-improvement for LLMs / [Model collapse from synthetic data](https://mlsanity.com/synth/model-collapse-from-synthetic-data/) ## Builds on (most cited first, 27 of 27) - [GPT-4 Technical Report](https://mlsanity.com/paper/2303.08774/) (2023) - [Llama 2: Open Foundation and Fine-Tuned Chat Models](https://mlsanity.com/paper/2307.09288/) (2023) - [Score-Based Generative Modeling through Stochastic Differential Equations](https://mlsanity.com/paper/2011.13456/) (2020) - [Hierarchical Text-Conditional Image Generation with CLIP Latents](https://mlsanity.com/paper/2204.06125/) (2022) - [Flow Matching for Generative Modeling](https://mlsanity.com/paper/2210.02747/) (2022) - [Auto-Encoding Variational Bayes](https://mlsanity.com/paper/1312.6114/) (2013) - [Flow Straight and Fast: Learning to Generate and Transfer Data with Rectified Flow](https://mlsanity.com/paper/2209.03003/) (2022) - [Progressive Growing of GANs for Improved Quality, Stability, and Variation](https://mlsanity.com/paper/1710.10196/) (2017) - [SDEdit: Guided Image Synthesis and Editing with Stochastic Differential Equations](https://mlsanity.com/paper/2108.01073/) (2021) - [Building Normalizing Flows with Stochastic Interpolants](https://mlsanity.com/paper/2209.15571/) (2022) - [Improving and generalizing flow-based generative models with minibatch optimal transport](https://mlsanity.com/paper/2302.00482/) (2023) - [WaveGrad: Estimating Gradients for Waveform Generation](https://mlsanity.com/paper/2009.00713/) (2020) - [Fast Sampling of Diffusion Models with Exponential Integrator](https://mlsanity.com/paper/2204.13902/) (2022) - [Consistency Trajectory Models: Learning Probability Flow ODE Trajectory of Diffusion](https://mlsanity.com/paper/2310.02279/) (2023) - [The Curse of Recursion: Training on Generated Data Makes Models Forget](https://mlsanity.com/paper/2305.17493/) (2023) - [Rectified Flow: A Marginal Preserving Approach to Optimal Transport](https://mlsanity.com/paper/2209.14577/) (2022) - [Multisample Flow Matching: Straightening Flows with Minibatch Couplings](https://mlsanity.com/paper/2304.14772/) (2023) - [Self-Consuming Generative Models Go MAD](https://mlsanity.com/paper/2307.01850/) (2023) - [Flow Matching in Latent Space](https://mlsanity.com/paper/2307.08698/) (2023) - [Is Model Collapse Inevitable? Breaking the Curse of Recursion by Accumulating Real and Synthetic Data](https://mlsanity.com/paper/2404.01413/) (2024) - [Improving the Training of Rectified Flows](https://mlsanity.com/paper/2405.20320/) (2024) - [Consistency Flow Matching: Defining Straight Flows with Velocity Consistency](https://mlsanity.com/paper/2407.02398/) (2024) - [On the Stability of Iterative Retraining of Generative Models on their own Data](https://mlsanity.com/paper/2310.00429/) (2023) - [Strong Model Collapse](https://mlsanity.com/paper/2410.04840/) (2024) - [Exact Diffusion Inversion via Bi-directional Integration Approximation](https://mlsanity.com/paper/2307.10829/) (2023) - [Combining Generative Artificial Intelligence (AI) and the Internet: Heading towards Evolution or Degradation?](https://mlsanity.com/paper/2303.01255/) (2023) - [Towards Theoretical Understandings of Self-Consuming Generative Models](https://mlsanity.com/paper/2402.11778/) (2024) ## Built on by (most cited first, 11 of 11) - [A Closer Look at Model Collapse: From a Generalization-to-Memorization Perspective](https://mlsanity.com/paper/2509.16499/) (2025) - [ReDi: Rectified Discrete Flow](https://mlsanity.com/paper/2507.15897/) (2025) - [Quantifying Error Propagation and Model Collapse in Diffusion Models](https://mlsanity.com/paper/2602.16601/) (2026) - [Low-Dimensional Adaptation of Rectified Flow: A Diffusion and Stochastic Localization Perspective](https://mlsanity.com/paper/2601.15500/) (2026) - [COT-FM: Cluster-wise Optimal Transport Flow Matching](https://mlsanity.com/paper/2603.13395/) (2026) - [Maximum Likelihood Estimation for Entity Ranking under Iterative Synthetic Data Augmentation](https://mlsanity.com/paper/2609.32987/) (2026) - [Flows, straight but not so fast: Exploring the design space of Rectified Flows in Protein Design](https://mlsanity.com/paper/2510.24732/) (2025) - [Recursively Trained Diffusion Models: Limiting Collapse Distribution and Spectral Characterization](https://mlsanity.com/paper/2606.13796/) (2026) - [ReFPO: Reflow Regularization for Flow Matching Policy Gradients](https://mlsanity.com/paper/2606.21086/) (2026) - [Optimal Self-Distillation for Rectified Flow via Linear Probing](https://mlsanity.com/paper/2607.14947/) (2026) - [Reviewing Model Collapse and Countermeasures](https://mlsanity.com/paper/2608.21366/) (2026) ## Abstract (arXiv) Training with synthetic data is becoming increasingly inevitable as synthetic content proliferates across the web, driven by the remarkable performance of recent deep generative models. This reliance on synthetic data can also be intentional, as seen in Rectified Flow models, whose Reflow method iteratively uses self-generated data to straighten the flow and improve sampling efficiency. However, recent studies have shown that repeatedly training on self-generated samples can lead to model collapse (MC), where performance degrades over time. Despite this, most recent work on MC either focuses on empirical observations or analyzes regression problems and maximum likelihood objectives, leaving a rigorous theoretical analysis of reflow methods unexplored. In this paper, we aim to fill this gap by providing both theoretical analysis and practical solutions for addressing MC in diffusion/flow models. We begin by studying Denoising Autoencoders and prove performance degradation when DAEs are iteratively trained on their own outputs. To the best of our knowledge, we are the first to rigorously analyze model collapse in DAEs and, by extension, in diffusion models and Rectified Flow. Our analysis and experiments demonstrate that rectified flow also suffers from MC, leading to potential performance degradation in each reflow step. Additionally, we prove that incorporating real data can prevent MC during recursive DAE training, supporting the recent trend of using real data as an effective approach for mitigating MC. Building on these insights, we propose a novel Real-data Augmented Reflow and a series of improved variants, which seamlessly integrate real data into Reflow training by leveraging reverse flow. Empirical evaluations on standard image benchmarks confirm that RA Reflow effectively mitigates model collapse, preserving high-quality sample generation even with fewer sampling steps.