mlsanityData to 7 Oct 2026

Exploring Continual Learning of Diffusion Models

arXiv 2303.15342 ↗First posted 27 Mar 2023cs.LG cs.AI cs.CVcited by 15

On the map: Diffusion models → Diffusion foundations and sampling

Abstract (arXiv)

Diffusion models have achieved remarkable success in generating high-quality images thanks to their novel training procedures applied to unprecedented amounts of data. However, training a diffusion model from scratch is computationally expensive. This highlights the need to investigate the possibility of training these models iteratively, reusing computation while the data distribution changes. In this study, we take the first step in this direction and evaluate the continual learning (CL) properties of diffusion models. We begin by benchmarking the most common CL methods applied to Denoising Diffusion Probabilistic Models (DDPMs), where we note the strong performance of the experience replay with the reduced rehearsal coefficient. Furthermore, we provide insights into the dynamics of forgetting, which exhibit diverse behavior across diffusion timesteps. We also uncover certain pitfalls of using the bits-per-dimension metric for evaluating CL.

Key facts. Cited by 15 papers in the map (data through 7 Oct 2026). Placed in Diffusion foundations and sampling, one of 14 directions in Diffusion models. Builds on 23 of its 28 references that are in the map. First posted on arXiv 27 Mar 2023.

Builds on

23 of 28 references.

Closest ancestors · 2

Same direction, newest first

  1. 2021Improved Denoising Diffusion Probabilistic Models18 Feb
  2. 2020Denoising Diffusion Probabilistic Models19 Jun

Other references · 21

  1. 2015Deep Unsupervised Learning using Nonequilibrium Thermodynamicscited by 7,005
  2. 2022Hierarchical Text-Conditional Image Generation with CLIP Latentscited by 6,539
  3. 2022Photorealistic Text-to-Image Diffusion Models with Deep Language Understandingcited by 6,322
  4. 2013Auto-Encoding Variational Bayescited by 3,980
  5. 2018Continual Lifelong Learning with Neural Networks: A Reviewcited by 1,313
  6. 2017Continual Learning with Deep Generative Replaycited by 1,152
  7. 2018Do Deep Generative Models Know What They Don't Know?cited by 443
  8. 2017Variational Continual Learningcited by 335
  9. 2015A note on the evaluation of generative modelscited by 291
  10. 2019Continual learning with hypernetworkscited by 267
  11. 2019Continual Unsupervised Representation Learningcited by 147
  12. 2019Lifelong GAN: Continual Learning for Conditional Image Generationcited by 78
9 more
  1. 2018Memory Replay GANs: learning to generate images from new categories without forgettingcited by 76
  2. 2021Rehearsal revealed: The limits and merits of revisiting samples in continual learningcited by 76
  3. 2018Generative Models from the perspective of Continual Learningcited by 69
  4. 2018Life-Long Disentangled Representation Learning with Cross-Domain Latent Homologiescited by 43
  5. 2022On the Effectiveness of Lipschitz-Driven Rehearsal in Continual Learningcited by 39
  6. 2020Learning latent representations across multiple data domains using Lifelong VAEGANcited by 35
  7. 2022On Analyzing Generative and Denoising Capabilities of Diffusion-based Deep Generative Modelscited by 31
  8. 2019BooVAE: Boosting Approach for Continual Learning of VAEcited by 18
  9. 2021Multiband VAE: Latent Space Alignment for Knowledge Consolidation in Continual Learningcited by 5

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 2; 2024: 9; 2025: 1; 2026: 3.

15 papers.

Sort:

By area:

  1. 2024Schr"odinger Bridge Flow for Unpaired Data Translationcited by 47
  2. 2023Continual Learning of Diffusion Models with Generative Distillationcited by 19
  3. 2024t-DGR: A Trajectory-Based Deep Generative Replay Method for Continual Learning in Decision Makingcited by 9
  4. 2024Joint Diffusion models in Continual Learningcited by 6
  5. 2024Stable Continual Reinforcement Learning via Diffusion-based Trajectory Replaycited by 6
  6. 2024GUIDE: Guidance-based Incremental Learning with Diffusion Modelscited by 5
  7. 2024CLoG: Benchmarking Continual Learning of Image Generation Modelscited by 5
  8. 2024CollaFuse: Collaborative Diffusion Modelscited by 4
  9. 2024Low-Rank Continual Personalization of Diffusion Modelscited by 3
  10. 2024Lifelong Learning of Video Diffusion Models From a Single Video Streamcited by 2
  11. 2026Forget Less by Learning Together through Concept Consolidationcited by 1
  12. 2023Adapt & Align: Continual Learning with Generative Models Latent Space Alignmentcited by 0
3 more
  1. 2025CCD: Continual Consistency Diffusion for Lifelong Generative Modelingcited by 0
  2. 2026Continual Learning in Modern Hopfield Networks with an Application to Diffusion Modelscited by 0
  3. 2026DriftScope: Measuring The Hidden Effects of Diffusion Model Adaptationcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Diffusion Models: A Mathematical Introduction
  2. 2026Memorization Control in Diffusion Models from Denoising-centric Perspective
  3. 2025Not Every Time and Frequency Need to Be Forgotten in Diffusion Unlearning
  4. 2025Quantifying the Ease of Reproducing Training Data in Unconditional Diffusion Models
  5. 2024Consistent Diffusion Meets Tweedie: Training Exact Ambient Diffusion Models with Noisy Data
  6. 2024Towards Faster Training of Diffusion Models: An Inspiration of A Consistency Phenomenon
  7. 2023The Missing U for Efficient Diffusion Models
  8. 2025Diffusion models under low-noise regime