mlsanityData to 7 Oct 2026

Self-Improving Diffusion Models with Synthetic Data

arXiv 2408.16333 ↗First posted 29 Aug 2024cs.LG cs.AIcited by 34

On the map: Diffusion models → Diffusion foundations and sampling

Abstract (arXiv)

The artificial intelligence (AI) world is running out of real data for training increasingly large generative models, resulting in accelerating pressure to train on synthetic data. Unfortunately, training new generative models with synthetic data from current or past generation models creates an autophagous (self-consuming) loop that degrades the quality and/or diversity of the synthetic data in what has been termed model autophagy disorder (MAD) and model collapse. Current thinking around model autophagy recommends that synthetic data is to be avoided for model training lest the system deteriorate into MADness. In this paper, we take a different tack that treats synthetic data differently from real data. Self-IMproving diffusion models with Synthetic data (SIMS) is a new training concept for diffusion models that uses self-synthesized data to provide negative guidance during the generation process to steer a model's generative process away from the non-ideal synthetic data manifold and towards the real data distribution. We demonstrate that SIMS is capable of self-improvement; it establishes new records based on the Fr'echet inception distance (FID) metric for CIFAR-10 and ImageNet-64 generation and achieves competitive results on FFHQ-64 and ImageNet-512. Moreover, SIMS is, to the best of our knowledge, the first prophylactic generative AI algorithm that can be iteratively trained on self-generated synthetic data without going MAD. As a bonus, SIMS can adjust a diffusion model's synthetic data distribution to match any desired in-domain target distribution to help mitigate biases and ensure fairness.

Key facts. Cited by 34 papers in the map (data through 7 Oct 2026). Placed in Diffusion foundations and sampling, one of 14 directions in Diffusion models. Builds on 31 of its 46 references that are in the map. First posted on arXiv 29 Aug 2024.

Builds on

31 of 46 references.

Closest ancestors · 14

Same direction, newest first

  1. 2024Guiding a Diffusion Model with a Bad Version of Itself4 Jun
  2. 2024Diffusion Models Are Innate One-Step Generators31 May
  3. 2024Applying Guidance in a Limited Interval Improves Sample and Distribution Quality in Diffusion Models11 Apr
  4. 2024Self-Rectifying Diffusion Sampling with Perturbed-Attention Guidance26 Mar
  5. 2023Analyzing and Improving the Training Dynamics of Diffusion Models5 Dec
  6. 2023End-to-End Diffusion Latent Optimization Improves Classifier Guidance23 Mar
  7. 2022Refining Generative Process with Discriminator Guidance in Score-based Diffusion Models28 Nov
  8. 2022Classifier-Free Diffusion Guidance26 Jul
  9. 2022Elucidating the Design Space of Diffusion-Based Generative Models1 Jun
  10. 2021High-Resolution Image Synthesis with Latent Diffusion Models20 Dec
  11. 2021Score-based Generative Modeling in Latent Space10 Jun
  12. 2021Diffusion Models Beat GANs on Image Synthesis11 May
2 more
  1. 2020Score-Based Generative Modeling through Stochastic Differential Equations26 Nov
  2. 2020Denoising Diffusion Probabilistic Models19 Jun

Other references · 17

  1. 2018A Style-Based Generator Architecture for Generative Adversarial Networkscited by 5,986
  2. 2019Generative Modeling by Estimating Gradients of the Data Distributioncited by 4,067
  3. 2020Training Generative Adversarial Networks with Limited Datacited by 1,184
  4. 2022StyleGAN-XL: Scaling StyleGAN to Large Diverse Datasetscited by 489
  5. 2023Self-Consuming Generative Models Go MADcited by 239
  6. 2022Scalable Adaptive Computation for Iterative Generationcited by 190
  7. 2023Fair Diffusion: Instructing Text-to-Image Generation Models on Fairnesscited by 149
  8. 2024Is Model Collapse Inevitable? Breaking the Curse of Recursion by Accumulating Real and Synthetic Datacited by 143
  9. 2024A Tale of Tails: Model Collapse as a Change of Scaling Lawscited by 127
  10. 2024RL on Incorrect Synthetic Data Scales the Efficiency of LLM Math Reasoning by Eight-Foldcited by 99
  11. 2023On the Stability of Iterative Retraining of Generative Models on their own Datacited by 98
  12. 2024Model Collapse Demystified: The Case of Regressioncited by 80
5 more
  1. 2024Fairness Feedback Loops: Training on Synthetic Data Amplifies Biascited by 55
  2. 2023RoSteALS: Robust Steganography using Autoencoder Latent Spacecited by 54
  3. 2023TrustMark: Universal Watermarking for Arbitrary Resolution Imagescited by 50
  4. 2024Self-Correcting Self-Consuming Loops for Generative Model Trainingcited by 34
  5. 2022Polarity Sampling: Quality and Diversity Control of Pre-Trained Generative Networks via Singular Valuescited by 21

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 4; 2025: 19; 2026: 11.

34 papers.

Sort:

By area:

  1. 2024Strong Model Collapsecited by 62
  2. 2025NoHumansRequired: Autonomous High-Quality Image Editing Triplet Miningcited by 33
  3. 2024Universality of the π2/6 Pathway in Avoiding Model Collapsecited by 24
  4. 2024Self-Guidance: Boosting Flow and Diffusion Generation on Their Owncited by 22
  5. 2025Ambient Diffusion Omni: Training Good Models with Bad Datacited by 18
  6. 2025A Closer Look at Model Collapse: From a Generalization-to-Memorization Perspectivecited by 18
  7. 2025A Theoretical Perspective: How to Prevent Model Collapse in Self-consuming Training Loopscited by 15
  8. 2025Machine-generated text detection prevents language model collapsecited by 14
  9. 2025EarthSynth: Generating Informative Earth Observation with Diffusion Modelscited by 12
  10. 2025PiSA: A Self-Augmented Data Engine and Training Strategy for 3D Understanding with Large Modelscited by 8
  11. 2025Delving into: the quantification of Ai-generated content on the internet (synthetic data)cited by 8
  12. 2025Self-Consuming Generative Models with Adversarially Curated Datacited by 8
22 more
  1. 2025Why Less is More (Sometimes): A Theory of Data Curationcited by 8
  2. 2025Self-Improving VLM Judges Without Human Annotationscited by 7
  3. 2025Increasing the Utility of Synthetic Images through Chamfer Guidancecited by 6
  4. 2025Neon: Negative Extrapolation From Self-Training Improves Image Generationcited by 5
  5. 2025Enhancing Virtual Try-On with Synthetic Pairs and Error-Aware Noise Schedulingcited by 3
  6. 2025Stabilizing Self-Consuming Diffusion Models with Latent Space Filteringcited by 3
  7. 2026Quantifying Error Propagation and Model Collapse in Diffusion Modelscited by 3
  8. 2024Guiding a diffusion model using sliding windowscited by 2
  9. 2025Noise-Level Diffusion Guidance: Well Begun is Half Donecited by 2
  10. 2025Beyond Real Data: Synthetic Data through the Lens of Regularizationcited by 2
  11. 2026Ambient Dataloops: Generative Models for Dataset Refinementcited by 2
  12. 2025ForTIFAI: Fending Off Recursive Training Induced Failure for AI Model Collapsecited by 1
  13. 2026Observations and Remedies for Large Language Model Bias in Self-Consuming Performative Loopcited by 1
  14. 2026Active Flow Expansion for Out-of-Distribution Discovery: from Theory to Moleculescited by 1
  15. 2025Overcoming the Curvature Bottleneck in MeanFlowcited by 0
  16. 2026Not All Synthetic Data Is Yours to Learn Fromcited by 0
  17. 2026ReSAGE-PAR: Representational Similarity Assessment for Generative Expansion in Pedestrian Attribute Recognitioncited by 0
  18. 2026Recursively Trained Diffusion Models: Limiting Collapse Distribution and Spectral Characterizationcited by 0
  19. 2026SynthSentry: Detecting Synthetic Data Contamination in Language Model Training Datacited by 0
  20. 2026Monotone-Constrained Diffusion Models for Long-Horizon Production Forecastingcited by 0
  21. 2026Improving Generative Model Self-Training with Geometrically Modified Outputscited by 0
  22. 2026ReGain: Restoring Subject Fidelity in Personalization on Synthetic Imagescited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026BlendFusion -- Scalable Synthetic Data Generation for Diffusion Model Training
  2. 2025Prototype-Guided Diffusion for Digital Pathology: Achieving Foundation Model Performance with Minimal Clinical Data
  3. 2024Are Images Indistinguishable to Humans Also Indistinguishable to Classifiers?
  4. 2024Simple Guidance Mechanisms for Discrete Diffusion Models
  5. 2023Diffusion Models for Generative Artificial Intelligence: An Introduction for Applied Mathematicians
  6. 2026Feature Selective Model Collapse in Diffusion Models: Total Replacement versus Fixed-Budget Training
  7. 2025Autoguided Online Data Curation for Diffusion Model Training
  8. 2025Diagnosing and Improving Diffusion Models by Estimating the Optimal Loss Value