mlsanityData to 14 Oct 2026

Class-Incremental Learning using Diffusion Model for Distillation and Replay

arXiv 2306.17560 ↗First posted 30 Jun 2023cs.LG cs.CVcited by 31

On the map: Diffusion models → Robustness, privacy and adversarial aspects

Abstract (arXiv)

Class-incremental learning aims to learn new classes in an incremental fashion without forgetting the previously learned ones. Several research works have shown how additional data can be used by incremental models to help mitigate catastrophic forgetting. In this work, following the recent breakthrough in text-to-image generative models and their wide distribution, we propose the use of a pretrained Stable Diffusion model as a source of additional data for class-incremental learning. Compared to competitive methods that rely on external, often unlabeled, datasets of real images, our approach can generate synthetic samples belonging to the same classes as the previously encountered images. This allows us to use those additional data samples not only in the distillation loss but also for replay in the classification loss. Experiments on the competitive benchmarks CIFAR100, ImageNet-Subset, and ImageNet demonstrate how this new approach can be used to further improve the performance of state-of-the-art methods for class-incremental learning on large scale datasets.

Key facts. Cited by 31 papers in the map (data through 14 Oct 2026). Placed in Robustness, privacy and adversarial aspects, one of 14 directions in Diffusion models. Builds on 25 of its 39 references that are in the map. First posted on arXiv 30 Jun 2023.

Builds on

25 of 39 references.

References · 25

  1. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  2. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,863
  3. 2014ImageNet Large Scale Visual Recognition Challengecited by 8,498
  4. 2022Classifier-Free Diffusion Guidancecited by 5,847
  5. 2016Learning without Forgettingcited by 2,496
  6. 2016iCaRL: Incremental Classifier and Representation Learningcited by 2,333
  7. 2017Continual Learning with Deep Generative Replaycited by 1,158
  8. 2019Large Scale Incremental Learningcited by 732
  9. 2018End-to-End Incremental Learningcited by 597
  10. 2019Dreaming to Distill: Data-free Knowledge Transfer via DeepInversioncited by 423
  11. 2021DER: Dynamically Expandable Representation for Class Incremental Learningcited by 355
  12. 2022Is synthetic data from generative models ready for image recognition?cited by 320
13 more
  1. 2022FOSTER: Feature Boosting and Compression for Class-Incremental Learningcited by 229
  2. 2020Mnemonics Training: Multi-Class Incremental Learning without Forgettingcited by 221
  3. 2023Class-Incremental Learning: A Surveycited by 209
  4. 2019Class-incremental Learning via Deep Model Consolidationcited by 173
  5. 2022Fake it till you make it: Learning transferable representations from synthetic ImageNet clonescited by 172
  6. 2021Always Be Dreaming: A New Approach for Data-Free Class-Incremental Learningcited by 131
  7. 2020SS-IL: Separated Softmax for Incremental Learningcited by 126
  8. 2021Distilling Causal Effect of Data in Class-Incremental Learningcited by 117
  9. 2019Overcoming Catastrophic Forgetting with Unlabeled Data in the Wildcited by 111
  10. 2021Generative Models as a Data Source for Multiview Representation Learningcited by 104
  11. 2021ORDisCo: Effective and Efficient Usage of Incremental Unlabeled Data for Semi-supervised Continual Learningcited by 68
  12. 2019This dataset does not exist: training models from generated imagescited by 65
  13. 2021Balanced softmax cross-entropy for incremental learning with and without memorycited by 13

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 2; 2024: 14; 2025: 9; 2026: 6.

31 papers.

Sort:

By area:

  1. 2024Boosting Continual Learning of Vision-Language Models via Mixture-of-Experts Adapterscited by 199
  2. 2023A Comprehensive Survey of Forgetting in Deep Learning Beyond Continual Learningcited by 69
  3. 2024SDDGR: Stable Diffusion-based Deep Generative Replay for Class Incremental Object Detectioncited by 44
  4. 2024DiffClass: Diffusion-Based Class Incremental Learningcited by 23
  5. 2024VLM-PL: Advanced Pseudo Labeling Approach for Class Incremental Object Detection via Vision-Language Modelcited by 23
  6. 2025Synthetic Data is an Elegant GIFT for Continual Vision-Language Modelscited by 21
  7. 2024Never-Ending Behavior-Cloning Agent for Robotic Manipulationcited by 10
  8. 2024GUIDE: Guidance-based Incremental Learning with Diffusion Modelscited by 5
  9. 2024Premonition: Using Generative Models to Preempt Future Data Changes in Continual Learningcited by 5
  10. 2024Future-Proofing Class-Incremental Learningcited by 4
  11. 2024Diffusion Model Meets Non-Exemplar Class-Incremental Learning and Beyondcited by 4
  12. 2024Low-Rank Continual Personalization of Diffusion Modelscited by 3
19 more
  1. 2025Divide-and-Conquer for Enhancing Unlabeled Learning, Stability, and Plasticity in Semi-supervised Continual Learningcited by 3
  2. 2023Prompt-Based Exemplar Super-Compression and Regeneration for Class-Incremental Learningcited by 2
  3. 2025Replay to Remember (R2R): An Efficient Uncertainty-driven Unsupervised Continual Learning Framework Using Generative Replaycited by 2
  4. 2025LoRA-Loop: Closing the Synthetic Replay Cycle for Continual VLM Learningcited by 2
  5. 2025Can Synthetic Images Conquer Forgetting? Beyond Unexplored Doubts in Few-Shot Class-Incremental Learningcited by 2
  6. 2024Exploiting Fine-Grained Prototype Distribution for Boosting Unsupervised Class Incremental Learningcited by 1
  7. 2025Beyond Synthetic Replays: Turning Diffusion Features into Few-Shot Class-Incremental Learning Knowledgecited by 1
  8. 2024Recommendation of data-free class-incremental learning algorithms by simulating future datacited by 0
  9. 2024Caption, Create, Continue: Continual Learning with Pre-trained Generative Vision-Language Modelscited by 0
  10. 2024Efficient Non-Exemplar Class-Incremental Learning with Retrospective Feature Synthesiscited by 0
  11. 2025Fresh-CL: Feature Realignment through Experts on Hypersphere in Continual Learningcited by 0
  12. 2025KDC-Diff: A Latent-Aware Diffusion Model with Knowledge Retention for Memory-Efficient Image Generationcited by 0
  13. 2025Deep Generative Continual Learning using Functional LoRA: FunLoRAcited by 0
  14. 2026Dream2Learn: Structured Generative Dreaming for Continual Learningcited by 0
  15. 2026Temporal Memory for Resource-Constrained Agents: Continual Learning via Stochastic Compress-Add-Smoothcited by 0
  16. 2026SR2-LoRA: Self-Rectifying Inter-layer Relations in Low-Rank Adaptation for Class-Incremental Learningcited by 0
  17. 2026Understanding Imbalanced Forgetting in Rehearsal-Based Class-Incremental Learningcited by 0
  18. 2026DRDN: Decoupled Representation Dynamic Network for From-Scratch ViT Class-Incremental Learningcited by 0
  19. 2026DR.WILSS: Diffusion-Based Replay for Weakly Supervised Continual Semantic Segmentationcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026Prototype Latent World Model Replay for Class-Incremental Learning
  2. 2023Class Incremental Learning with Self-Supervised Pre-Training and Prototype Learning
  3. 2025Breaking Forgetting: Training-Free Few-Shot Class-Incremental Learning via Conditional Diffusion
  4. 2021Dual-Teacher Class-Incremental Learning With Data-Free Generative Replay
  5. 2022Class-Incremental Learning by Knowledge Distillation with Adaptive Feature Consolidation
  6. 2024Data-Free Federated Class Incremental Learning with Diffusion-Based Generative Memory
  7. 2023Class-Prototype Conditional Diffusion Model with Gradient Projection for Continual Learning
  8. 2023SRIL: Selective Regularization for Class-Incremental Learning