mlsanityData to 14 Oct 2026

Generative Negative Replay for Continual Learning

arXiv 2204.05842 ↗First posted 12 Apr 2022cs.LG cs.CV stat.MLcited by 4

Not on the map. Why?

Abstract (arXiv)

Learning continually is a key aspect of intelligence and a necessary ability to solve many real-life problems. One of the most effective strategies to control catastrophic forgetting, the Achilles' heel of continual learning, is storing part of the old data and replaying them interleaved with new experiences (also known as the replay approach). Generative replay, which is using generative models to provide replay patterns on demand, is particularly intriguing, however, it was shown to be effective mainly under simplified assumptions, such as simple scenarios and low-dimensional data. In this paper, we show that, while the generated data are usually not able to improve the classification accuracy for the old classes, they can be effective as negative examples (or antagonists) to better learn the new classes, especially when the learning experiences are small and contain examples of just one or few classes. The proposed approach is validated on complex class-incremental and data-incremental continual learning scenarios (CORe50 and ImageNet-1000) composed of high-dimensional data and a large number of training experiences: a setup where existing generative replay approaches usually fail.

Key facts. Cited by 4 papers in the map (data through 14 Oct 2026). Builds on 37 of its 61 references that are in the map. First posted on arXiv 12 Apr 2022.

Builds on

37 of 61 references.

References · 37

  1. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  2. 2018A Style-Based Generator Architecture for Generative Adversarial Networkscited by 6,013
  3. 2022Generative Adversarial Networkscited by 5,627
  4. 2016Overcoming catastrophic forgetting in neural networkscited by 4,787
  5. 2013Auto-Encoding Variational Bayescited by 4,053
  6. 2017MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applicationscited by 3,845
  7. 2020Supervised Contrastive Learningcited by 2,829
  8. 2016Learning without Forgettingcited by 2,496
  9. 2016iCaRL: Incremental Classifier and Representation Learningcited by 2,333
  10. 2018Continual Lifelong Learning with Neural Networks: A Reviewcited by 1,330
  11. 2017Continual Learning with Deep Generative Replaycited by 1,158
  12. 2018Riemannian Walk for Incremental Learning: Understanding Forgetting and Intransigencecited by 742
25 more
  1. 2019Large Scale Incremental Learningcited by 732
  2. 2019Three scenarios for continual learningcited by 614
  3. 2018End-to-End Incremental Learningcited by 597
  4. 2019Gradient based sample selection for online continual learningcited by 571
  5. 2016Conditional Image Synthesis With Auxiliary Classifier GANscited by 565
  6. 2020Class-incremental learning: survey and performance evaluation on image classificationcited by 449
  7. 2017CORe50: a New Dataset and Benchmark for Continuous Object Recognitioncited by 289
  8. 2018Learning without Memorizingcited by 250
  9. 2020Supermasks in Superpositioncited by 227
  10. 2019Learning to Remember: A Synaptic Plasticity Driven Framework for Continual Learningcited by 182
  11. 2017FearNet: Brain-Inspired Model for Incremental Learningcited by 177
  12. 2017Batch Renormalization: Towards Reducing Minibatch Dependence in Batch-Normalized Modelscited by 128
  13. 2018Continuous Learning in Single-Incremental-Task Scenarioscited by 109
  14. 2020A Comprehensive Study of Class Incremental Learning Algorithms for Visual Taskscited by 107
  15. 2019Latent Replay for Real-Time Continual Learningcited by 97
  16. 2019Lifelong Machine Learning with Deep Streaming Linear Discriminant Analysiscited by 92
  17. 2018IntroVAE: Introspective Variational Autoencoders for Photographic Image Synthesiscited by 85
  18. 2019Lifelong GAN: Continual Learning for Conditional Image Generationcited by 78
  19. 2018Memory Replay GANs: learning to generate images from new categories without forgettingcited by 77
  20. 2021Replay in Deep Learning: Current Approaches and Missing Biological Elementscited by 72
  21. 2018Generative Models from the perspective of Continual Learningcited by 69
  22. 2019Rehearsal-Free Continual Learning over Small Non-I.I.D. Batchescited by 46
  23. 2021EEC: Learning to Encode and Regenerate Images for Continual Learningcited by 40
  24. 2019Unified Probabilistic Deep Continual Learning through Generative Replay and Open Set Recognitioncited by 26
  25. 2021Continual Learning at the Edge: Real-Time Training on Smartphone Devicescited by 20

Built on by

Cited by year

Citing papers per year: 2023: 2; 2024: 1; 2025: 1.

4 papers.

  1. 2025The Future of Continual Learning in the Era of Foundation Models: Three Key Directionscited by 17
  2. 2023Partial Hypernetworks for Continual Learningcited by 7
  3. 2023Architect, Regularize and Replay (ARR): a Flexible Hybrid Approach for Continual Learningcited by 1
  4. 2024Continual Learning by Three-Phase Consolidationcited by 0