mlsanityData to 7 Oct 2026

REMIND Your Neural Network to Prevent Catastrophic Forgetting

arXiv 1910.02509 ↗First posted 6 Oct 2019cs.LG cs.CV cs.NEcited by 197

Not on the map. Why?

Abstract (arXiv)

People learn throughout life. However, incrementally updating conventional neural networks leads to catastrophic forgetting. A common remedy is replay, which is inspired by how the brain consolidates memory. Replay involves fine-tuning a network on a mixture of new and old instances. While there is neuroscientific evidence that the brain replays compressed memories, existing methods for convolutional networks replay raw images. Here, we propose REMIND, a brain-inspired approach that enables efficient replay with compressed representations. REMIND is trained in an online manner, meaning it learns one example at a time, which is closer to how humans learn. Under the same constraints, REMIND outperforms other methods for incremental class learning on the ImageNet ILSVRC-2012 dataset. We probe REMIND's robustness to data ordering schemes known to induce catastrophic forgetting. We demonstrate REMIND's generality by pioneering online learning for Visual Question Answering (VQA).

Key facts. Cited by 197 papers in the map (data through 7 Oct 2026). First posted on arXiv 6 Oct 2019.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 31; 2022: 38; 2023: 50; 2024: 43; 2025: 20; 2026: 15.

197 papers.

Sort:

By area:

  1. 2023A Comprehensive Survey of Continual Learning: Theory, Method and Applicationcited by 860
  2. 2021DyTox: Transformers for Continual Learning with DYnamic TOken eXpansioncited by 291
  3. 2021Rainbow Memory: Continual Learning with a Memory of Diverse Samplescited by 273
  4. 2021Online Continual Learning in Image Classification: An Empirical Surveycited by 242
  5. 2023SLCA: Slow Learner with Classifier Alignment for Continual Learning on a Pre-trained Modelcited by 178
  6. 2023Continual Diffusion: Continual Customization of Text-to-Image Diffusion with C-LoRAcited by 138
  7. 2024Vaccine: Perturbation-aware Alignment for Large Language Models against Harmful Fine-tuning Attackcited by 113
  8. 2022Memory Replay with Data Compression for Continual Learningcited by 80
  9. 2024Continual Learning and Catastrophic Forgettingcited by 73
  10. 2022Representation Compensation Networks for Continual Semantic Segmentationcited by 72
  11. 2021Replay in Deep Learning: Current Approaches and Missing Biological Elementscited by 71
  12. 2022Continual Learning with Foundation Models: An Empirical Study of Latent Replaycited by 61
38 more
  1. 2021Recent Advances of Continual Learning in Computer Vision: An Overviewcited by 55
  2. 2021Self-Supervised Training Enhances Online Continual Learningcited by 53
  3. 2024Addressing Loss of Plasticity and Catastrophic Forgetting in Continual Learningcited by 49
  4. 2022Online Continual Learning for Embedded Devicescited by 48
  5. 2022PIVOT: Prompting for Video Continual Learningcited by 45
  6. 2021Class-Incremental Learning with Generative Classifierscited by 39
  7. 2021DualNet: Continual Learning, Fast and Slowcited by 39
  8. 2024Recent Advances of Multimodal Continual Learning: A Comprehensive Surveycited by 39
  9. 2021Continual Learning for Blind Image Quality Assessmentcited by 36
  10. 2022Long-Tailed Class Incremental Learningcited by 36
  11. 2023The Janus Interface: How Fine-Tuning in Large Language Models Amplifies the Privacy Riskscited by 35
  12. 2022SBERT studies Meaning Representations: Decomposing Sentence Embeddings into Explainable Semantic Featurescited by 33
  13. 2024Convolutional Prompting meets Language Models for Continual Learningcited by 32
  14. 2023Online Continual Learning Without the Storage Constraintcited by 29
  15. 2022A Multi-Head Model for Continual Learning via Out-of-Distribution Replaycited by 27
  16. 2022Continual Learning with Bayesian Model based on a Fixed Pre-trained Feature Extractorcited by 26
  17. 2024Online-LoRA: Task-free Online Continual Learning via Low Rank Adaptationcited by 26
  18. 2023BiRT: Bio-inspired Replay in Vision Transformers for Continual Learningcited by 25
  19. 2023SIESTA: Efficient Online Continual Learning with Sleepcited by 24
  20. 2023How Efficient Are Today's Continual Learning Algorithms?cited by 24
  21. 2023Dealing with Cross-Task Class Discrimination in Online Continual Learningcited by 24
  22. 2023Catastrophic Forgetting in Deep Learning: A Comprehensive Taxonomycited by 24
  23. 2024Generative Multi-modal Models are Good Class-Incremental Learnerscited by 22
  24. 2021Continual Learning in Sensor-based Human Activity Recognition: an Empirical Benchmark Analysiscited by 21
  25. 2022Continual Learning Based on OOD Detection and Task Maskingcited by 21
  26. 2023Continual Segment: Towards a Single, Unified and Accessible Continual Segmentation Model of 143 Whole-body Organs in CT Scanscited by 21
  27. 2021Condensed Composite Memory Continual Learningcited by 20
  28. 2022Challenging Common Assumptions about Catastrophic Forgettingcited by 20
  29. 2022Do Pre-trained Models Benefit Equally in Continual Learning?cited by 20
  30. 2022ConStruct-VL: Data-Free Continual Structured VL Concepts Learningcited by 20
  31. 2023Unlocking the Potential of Prompt-Tuning in Bridging Generalized and Personalized Federated Learningcited by 20
  32. 2021Preserving Earlier Knowledge in Continual Learning with the Help of All Previous Feature Extractorscited by 19
  33. 2021On the importance of cross-task features for class-incremental learningcited by 18
  34. 2022Semantic Segmentation with Active Semi-Supervised Learningcited by 16
  35. 2023New Insights for the Stability-Plasticity Dilemma in Online Continual Learningcited by 16
  36. 2021Tackling Catastrophic Forgetting and Background Shift in Continual Semantic Segmentationcited by 15
  37. 2023Class Incremental Learning with Pre-trained Vision-Language Modelscited by 15
  38. 2024Memorization in deep learning: A surveycited by 14

50 most cited of 197 All 197 (JSON)