mlsanityData to 7 Oct 2026

Toward Understanding Catastrophic Forgetting in Continual Learning

arXiv 1908.01091 ↗First posted 2 Aug 2019cs.LG cs.CV stat.MLcited by 67

Not on the map. Why?

Abstract (arXiv)

We study the relationship between catastrophic forgetting and properties of task sequences. In particular, given a sequence of tasks, we would like to understand which properties of this sequence influence the error rates of continual learning algorithms trained on the sequence. To this end, we propose a new procedure that makes use of recent developments in task space modeling as well as correlation analysis to specify and analyze the properties we are interested in. As an application, we apply our procedure to study two properties of a task sequence: (1) total complexity and (2) sequential heterogeneity. We show that error rates are strongly and positively correlated to a task sequence's total complexity for some state-of-the-art algorithms. We also show that, surprisingly, the error rates have no or even negative correlations in some cases to sequential heterogeneity. Our findings suggest directions for improving continual learning benchmarks and methods.

Key facts. Cited by 67 papers in the map (data through 7 Oct 2026). First posted on arXiv 2 Aug 2019.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 6; 2022: 16; 2023: 7; 2024: 14; 2025: 15; 2026: 9.

67 papers.

Sort:

By area:

  1. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,865
  2. 2022CODA-Prompt: COntinual Decomposed Attention-based Prompting for Rehearsal-Free Continual Learningcited by 446
  3. 2021Continual Learning in the Teacher-Student Setup: Impact of Task Similaritycited by 86
  4. 2022Probing Representation Forgetting in Supervised and Unsupervised Continual Learningcited by 74
  5. 2021Wide Neural Networks Forget Less Catastrophicallycited by 64
  6. 2022Continual Learning with Foundation Models: An Empirical Study of Latent Replaycited by 61
  7. 2022A Review of Deep Transfer Learning and Recent Advancementscited by 56
  8. 2022A Closer Look at Rehearsal-Free Continual Learningcited by 54
  9. 2024Self-Expansion of Pre-trained Models with Mixture of Adapters for Continual Learningcited by 53
  10. 2024Mixture of Experts Meets Prompt-Based Continual Learningcited by 43
  11. 2022The Effect of Task Ordering in Continual Learningcited by 30
  12. 2024Unleashing the Power of Data Tsunami: A Comprehensive Survey on Data Assessment and Selection for Instruction Tuning of Language Modelscited by 29
38 more
  1. 2022Wasserstein Task Embedding for Measuring Task Similaritiescited by 24
  2. 2025DA-VPT: Semantic-Guided Visual Prompt Tuning for Vision Transformerscited by 17
  3. 2024iKAN: Global Incremental Learning with KAN for Human Activity Recognition Across Heterogeneous Datasetscited by 14
  4. 2024Towards Effective and Efficient Continual Pre-training of Large Language Modelscited by 13
  5. 2025Continual Unlearning for Foundational Text-to-Image Models without Generalization Erosioncited by 11
  6. 2025Through the Valley: Path to Effective Long CoT Training for Small Language Modelscited by 10
  7. 2024EVCL: Elastic Variational Continual Learning with Weight Consolidationcited by 9
  8. 2022Theoretical Understanding of the Information Flow on Continual Learning Performancecited by 8
  9. 2024Predicting the Susceptibility of Examples to Catastrophic Forgettingcited by 7
  10. 2023L3DMC: Lifelong Learning using Distillation via Mixed-Curvature Spacecited by 6
  11. 2025Continual Learning on CLIP via Incremental Prompt Tuning with Intrinsic Textual Anchorscited by 6
  12. 2021On robustness of generative representations against catastrophic forgettingcited by 5
  13. 2021Dendritic Self-Organizing Maps for Continual Learningcited by 5
  14. 2023Measuring and Mitigating Interference in Reinforcement Learningcited by 5
  15. 2024Continual Learning Improves Zero-Shot Action Recognitioncited by 5
  16. 2025Sequence Transferability and Task Order Selection in Continual Learningcited by 5
  17. 2022Generalization Bounds for Deep Transfer Learning Using Majority Predictor Accuracycited by 4
  18. 2022Causes of Catastrophic Forgetting in Class-Incremental Semantic Segmentationcited by 4
  19. 2022Task Formulation Matters When Learning Continually: A Case Study in Visual Question Answeringcited by 4
  20. 2022Navigating Memory Construction by Global Pseudo-Task Simulation for Continual Learningcited by 4
  21. 2023Principles of Forgetting in Domain-Incremental Semantic Segmentation in Adverse Weather Conditionscited by 4
  22. 2025Little by Little: Continual Learning via Incremental Mixture of Rank-1 Associative Memory Expertscited by 4
  23. 2021Model architecture can transform catastrophic forgetting into positive transfercited by 3
  24. 2025DrugMCTS: a drug repurposing framework combining multi-agent, RAG and Monte Carlo Tree Searchcited by 3
  25. 2022Lifelong Self-Adaptation: Self-Adaptation Meets Lifelong Machine Learningcited by 2
  26. 2022Dealing with Drift of Adaptation Spaces in Learning-based Self-Adaptive Systems using Lifelong Self-Adaptationcited by 2
  27. 2023ScriptWorld: Text Based Environment For Learning Procedural Knowledgecited by 2
  28. 2023Rethinking E-Commerce Searchcited by 2
  29. 2024FlipGuard: Defending Preference Alignment against Update Regression with Constrained Optimizationcited by 2
  30. 2024Using Diffusion Models as Generative Replay in Continual Federated Learning -- What will Happen?cited by 2
  31. 2024Take Only What You Need: Rank Minimization as an Implicit Forgetting Regularizer in Continual Learningcited by 2
  32. 2025One-Prompt Strikes Back: Sparse Mixture of Experts for Prompt-based Continual Learningcited by 2
  33. 2021Dynamic neighbourhood optimisation for task allocation using multi-agentcited by 1
  34. 2024Can Similarity-Based Domain-Ordering Reduce Catastrophic Forgetting for Intent Recognition?cited by 1
  35. 2025Multiple Queries with Multiple Keys: A Precise Prompt Matching Paradigm for Prompt-based Continual Learningcited by 1
  36. 2025Should Code Models Learn Pedagogically? A Preliminary Evaluation of Curriculum Learning for Real-World Software Engineering Taskscited by 1
  37. 2025WAVE++: Capturing Within-Task Variance for Continual Relation Extraction with Adaptive Promptingcited by 1
  38. 2025Towards Robust Visual Continual Learning with Multi-Prototype Supervisioncited by 1

50 most cited of 67 All 67 (JSON)