mlsanityData to 7 Oct 2026

Continual Learning with Deep Generative Replay

arXiv 1705.08690 ↗First posted 24 May 2017cs.AI cs.CV cs.LGcited by 1,152

Not on the map. Why?

Cited by 27 in Diffusion models, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Attempts to train a comprehensive artificial intelligence capable of solving multiple tasks have been impeded by a chronic problem called catastrophic forgetting. Although simply replaying all previous data alleviates the problem, it requires large memory and even worse, often infeasible in real world applications where the access to past data is limited. Inspired by the generative nature of hippocampus as a short-term memory system in primate brain, we propose the Deep Generative Replay, a novel framework with a cooperative dual model architecture consisting of a deep generative model ("generator") and a task solving model ("solver"). With only these two models, training data for previous tasks can easily be sampled and interleaved with those for a new task. We test our methods in several sequential learning settings involving image classification tasks.

Key facts. Cited by 1,152 papers in the map (data through 7 Oct 2026). First posted on arXiv 24 May 2017.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 197; 2022: 194; 2023: 218; 2024: 230; 2025: 206; 2026: 107.

1,152 papers.

Sort:

By area:

  1. 2023A Comprehensive Survey of Continual Learning: Theory, Method and Applicationcited by 860
  2. 2021iMAP: Implicit Mapping and Positioning in Real-Timecited by 476
  3. 2022CODA-Prompt: COntinual Decomposed Attention-based Prompting for Rehearsal-Free Continual Learningcited by 446
  4. 2023A Comprehensive Survey on Test-Time Adaptation under Distribution Shiftscited by 353
  5. 2021Gradient Projection Memory for Continual Learningcited by 338
  6. 2021DyTox: Transformers for Continual Learning with DYnamic TOken eXpansioncited by 291
  7. 2021Rainbow Memory: Continual Learning with a Memory of Diverse Samplescited by 273
  8. 2021Training Spiking Neural Networks Using Lessons From Deep Learningcited by 251
  9. 2022S-Prompts Learning with Pre-trained Transformers: An Occam's Razor for Domain Incremental Learningcited by 246
  10. 2021Online Continual Learning in Image Classification: An Empirical Surveycited by 242
  11. 2023Class-Incremental Learning: A Surveycited by 209
  12. 2024Boosting Continual Learning of Vision-Language Models via Mixture-of-Experts Adapterscited by 199
38 more
  1. 2023Continual Pre-training of Language Modelscited by 194
  2. 2022Patching open-vocabulary models by interpolating weightscited by 187
  3. 2021Few-Shot Incremental Learning with Continually Evolved Classifierscited by 184
  4. 2023Revisiting Class-Incremental Learning with Pre-Trained Models: Generalizability and Adaptivity are All You Needcited by 183
  5. 2023Mitigating the Alignment Tax of RLHFcited by 181
  6. 2023Selective Amnesia: A Continual Learning Approach to Forgetting in Deep Generative Modelscited by 175
  7. 2023One-Dimensional Adapter to Rule Them All: Concepts, Diffusion Models and Erasing Applicationscited by 164
  8. 2022Fine-tuned Language Models are Continual Learnerscited by 152
  9. 2022A Model or 603 Exemplars: Towards Memory-Efficient Class-Incremental Learningcited by 152
  10. 2021Self-Supervised Models are Continual Learnerscited by 149
  11. 2024Continual Learning for Large Language Models: A Surveycited by 144
  12. 2021Training Networks in Null Space of Feature Covariance for Continual Learningcited by 139
  13. 2023Continual Diffusion: Continual Customization of Text-to-Image Diffusion with C-LoRAcited by 138
  14. 2021An Empirical Investigation of the Role of Pre-training in Lifelong Learningcited by 134
  15. 2023Domain Specialization as the Key to Make Large Language Models Disruptive: A Comprehensive Surveycited by 134
  16. 2021Always Be Dreaming: A New Approach for Data-Free Class-Incremental Learningcited by 130
  17. 2022Federated Class-Incremental Learningcited by 126
  18. 2022Continual Learning and Private Unlearningcited by 124
  19. 2023A Unified Continual Learning Framework with General Parameter-Efficient Tuningcited by 124
  20. 2021Distilling Causal Effect of Data in Class-Incremental Learningcited by 117
  21. 2021Semantic-aware Knowledge Distillation for Few-Shot Class-Incremental Learningcited by 114
  22. 2022Class-Incremental Learning by Knowledge Distillation with Adaptive Feature Consolidationcited by 113
  23. 2023Preventing Zero-Shot Transfer Degradation in Continual Learning of Vision-Language Modelscited by 112
  24. 2021Rethinking Architecture Design for Tackling Data Heterogeneity in Federated Learningcited by 111
  25. 2023Synthetic Experience Replaycited by 100
  26. 2021Continual Semantic Segmentation via Repulsion-Attraction of Sparse and Disentangled Latent Representationscited by 96
  27. 2024Revisiting Catastrophic Forgetting in Large Language Model Tuningcited by 93
  28. 2021Achieving Forgetting Prevention and Knowledge Transfer in Continual Learningcited by 91
  29. 2021Continual Learning for Text Classification with Information Disentanglement Based Regularizationcited by 86
  30. 2021Invariant Information Bottleneck for Domain Generalizationcited by 86
  31. 2021Continual Learning in the Teacher-Student Setup: Impact of Task Similaritycited by 86
  32. 2021On Learning the Geodesic Path for Incremental Learningcited by 85
  33. 2022Wild-Time: A Benchmark of in-the-Wild Distribution Shift over Timecited by 85
  34. 2021RECALL: Replay-based Continual Learning in Semantic Segmentationcited by 83
  35. 2022Continual Learning of Natural Language Processing Tasks: A Surveycited by 83
  36. 2023Circumventing Concept Erasure Methods For Text-to-Image Generative Modelscited by 81
  37. 2021Continual Learning of a Mixed Sequence of Similar and Dissimilar Taskscited by 80
  38. 2022Memory Replay with Data Compression for Continual Learningcited by 80

50 most cited of 1,152 All 1,152 (JSON)