mlsanityData to 7 Oct 2026

A comprehensive, application-oriented study of catastrophic forgetting in DNNs

arXiv 1905.08101 ↗First posted 20 May 2019cs.LG stat.MLcited by 35

Not on the map. Why?

Abstract (arXiv)

We present a large-scale empirical study of catastrophic forgetting (CF) in modern Deep Neural Network (DNN) models that perform sequential (or: incremental) learning. A new experimental protocol is proposed that enforces typical constraints encountered in application scenarios. As the investigation is empirical, we evaluate CF behavior on the hitherto largest number of visual classification datasets, from each of which we construct a representative number of Sequential Learning Tasks (SLTs) in close alignment to previous works on CF. Our results clearly indicate that there is no model that avoids CF for all investigated datasets and SLTs under application conditions. We conclude with a discussion of potential solutions and workarounds to CF, notably for the EWC and IMM models.

Key facts. Cited by 35 papers in the map (data through 7 Oct 2026). First posted on arXiv 20 May 2019.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 13; 2022: 8; 2023: 6; 2024: 5; 2025: 2; 2026: 1.

35 papers.

Sort:

  1. 2021Online Continual Learning in Image Classification: An Empirical Surveycited by 242
  2. 2022Open-environment Machine Learningcited by 48
  3. 2021Continually Learning Self-Supervised Representations with Projected Functional Regularizationcited by 38
  4. 2022On the Soft-Subnetwork for Few-shot Class Incremental Learningcited by 37
  5. 2023Online Class Incremental Learning on Stochastic Blurry Task Boundary via Mask and Visual Prompt Tuningcited by 35
  6. 2021CLEVA-Compass: A Continual Learning EValuation Assessment Compass to Promote Research Transparency and Comparabilitycited by 28
  7. 2022Analysis of Catastrophic Forgetting for Random Orthogonal Transformation Tasks in the Overparameterized Regimecited by 28
  8. 2021An Investigation of Replay-based Approaches for Continual Learningcited by 26
  9. 2024The Joint Effect of Task Similarity and Overparameterization on Catastrophic Forgetting -- An Analytical Modelcited by 26
  10. 2023Catastrophic Forgetting in Deep Learning: A Comprehensive Taxonomycited by 24
  11. 2021Continual Learning in Sensor-based Human Activity Recognition: an Empirical Benchmark Analysiscited by 21
  12. 2021RoRD: Rotation-Robust Descriptors and Orthographic Views for Local Feature Matchingcited by 19
23 more
  1. 2023Retentive or Forgetful? Diving into the Knowledge Memorizing Mechanism of Language Modelscited by 18
  2. 2024Towards Robust Graph Incremental Learning on Evolving Graphscited by 18
  3. 2021Exploring System Performance of Continual Learning for Mobile and Embedded Sensing Applicationscited by 17
  4. 2022Provable and Efficient Continual Representation Learningcited by 14
  5. 2022Beyond Supervised Continual Learning: a Reviewcited by 11
  6. 2021FastICARL: Fast Incremental Classifier and Representation Learning with Efficient Budget Allocation in Audio Sensing Applicationscited by 9
  7. 2021Overcoming Catastrophic Forgetting with Gaussian Mixture Replaycited by 7
  8. 2024Predicting the Susceptibility of Examples to Catastrophic Forgettingcited by 7
  9. 2021Continual Learning with Fully Probabilistic Modelscited by 6
  10. 2021A Procedural World Generation Framework for Systematic Evaluation of Continual Learningcited by 4
  11. 2022A Study of Continual Learning Methods for Q-Learningcited by 3
  12. 2023Adiabatic replay for continual learningcited by 3
  13. 2024VIPeR: Visual Incremental Place Recognition with Adaptive Mining and Continual Learningcited by 2
  14. 2021Reviewing continual learning from the perspective of human-level intelligencecited by 1
  15. 2023Benchmarking Continual Learning from Cognitive Perspectivescited by 1
  16. 2024Variational Bayes for Federated Continual Learningcited by 1
  17. 2025The Bayesian Approach to Continual Learning: An Overviewcited by 1
  18. 2021Overcome Anterograde Forgetting with Cycled Memory Networkscited by 0
  19. 2022The Multiple Subnetwork Hypothesis: Enabling Multidomain Learning by Isolating Task-Specific Subnetworks in Feedforward Neural Networkscited by 0
  20. 2022TegFormer: Topic-to-Essay Generation with Good Topic Coverage and High Text Coherencecited by 0
  21. 2023CSSL-MHTR: Continual Self-Supervised Learning for Scalable Multi-script Handwritten Text Recognitioncited by 0
  22. 2025A Benchmark for Incremental Micro-expression Recognitioncited by 0
  23. 2026Why Do Neural Networks Forget: A Study of Collapse in Continual Learningcited by 0