mlsanityData to 7 Oct 2026

Task Confusion and Catastrophic Forgetting in Class-Incremental Learning: A Mathematical Framework for Discriminative and Generative Modelings

arXiv 2410.20768 ↗First posted 28 Oct 2024cs.LGcited by 4

Not on the map. Why?

Abstract (arXiv)

In class-incremental learning (class-IL), models must classify all previously seen classes at test time without task-IDs, leading to task confusion. Despite being a key challenge, task confusion lacks a theoretical understanding. We present a novel mathematical framework for class-IL and prove the Infeasibility Theorem, showing optimal class-IL is impossible with discriminative modeling due to task confusion. However, we establish the Feasibility Theorem, demonstrating that generative modeling can achieve optimal class-IL by overcoming task confusion. We then assess popular class-IL strategies, including regularization, bias-correction, replay, and generative classifier, using our framework. Our analysis suggests that adopting generative modeling, either for generative replay or direct classification (generative classifier), is essential for optimal class-IL.

Key facts. Cited by 4 papers in the map (data through 7 Oct 2026). Builds on 40 of its 53 references that are in the map. First posted on arXiv 28 Oct 2024.

Builds on

40 of 53 references.

References · 40

  1. 2016Overcoming catastrophic forgetting in neural networkscited by 4,718
  2. 2013Auto-Encoding Variational Bayescited by 3,980
  3. 2016iCaRL: Incremental Classifier and Representation Learningcited by 2,313
  4. 2020Shortcut Learning in Deep Neural Networkscited by 1,928
  5. 2017Continual Learning with Deep Generative Replaycited by 1,152
  6. 2018Experience Replay for Continual Learningcited by 1,074
  7. 2019Large Scale Incremental Learningcited by 727
  8. 2018Overcoming catastrophic forgetting with hard attention to the taskcited by 672
  9. 2019Three scenarios for continual learningcited by 610
  10. 2018End-to-End Incremental Learningcited by 596
  11. 2015Importance Weighted Autoencoderscited by 454
  12. 2020Class-incremental learning: survey and performance evaluation on image classificationcited by 446
28 more
  1. 2019Online Continual Learning with Maximally Interfered Retrievalcited by 376
  2. 2017Variational Continual Learningcited by 335
  3. 2017CORe50: a New Dataset and Benchmark for Continuous Object Recognitioncited by 287
  4. 2018Learning without Memorizingcited by 249
  5. 2020Semantic Drift Compensation for Class-Incremental Learningcited by 241
  6. 2018Re-evaluating Continual Learning Scenarios: A Categorization and Case for Strong Baselinescited by 228
  7. 2018Task-Free Continual Learningcited by 219
  8. 2019REMIND Your Neural Network to Prevent Catastrophic Forgettingcited by 197
  9. 2019Class-incremental Learning via Deep Model Consolidationcited by 172
  10. 2018Towards Robust Evaluations of Continual Learningcited by 148
  11. 2020Continual Prototype Evolution: Learning Online from Non-Stationary Data Streamscited by 143
  12. 2018Continuous Learning in Single-Incremental-Task Scenarioscited by 109
  13. 2020Generative Feature Replay For Class-Incremental Learningcited by 109
  14. 2020A Comprehensive Study of Class Incremental Learning Algorithms for Visual Taskscited by 107
  15. 2018Generative replay with feedback connections as a general strategy for continual learningcited by 103
  16. 2019Lifelong Machine Learning with Deep Streaming Linear Discriminant Analysiscited by 91
  17. 2020A Wholistic View of Continual Learning with Deep Neural Networks: Forgotten Lessons and the Bridge to Active and Open World Learningcited by 86
  18. 2020GAN Memory with No Forgettingcited by 84
  19. 2022A Theoretical Study on Solving Continual Learningcited by 78
  20. 2018Generative Models from the perspective of Continual Learningcited by 69
  21. 2020ScaIL: Classifier Weights Scaling for Class Incremental Learningcited by 47
  22. 2021Natural continual learning: success is a journey, not (just) a destinationcited by 46
  23. 2021Class-Incremental Learning with Generative Classifierscited by 39
  24. 2019A Unifying Bayesian View of Continual Learningcited by 38
  25. 2020Task Agnostic Continual Learning Using Online Variational Bayes with Fixed-Point Updatescited by 34
  26. 2020Energy-Based Models for Continual Learningcited by 32
  27. 2021On the importance of cross-task features for class-incremental learningcited by 18
  28. 2022Chroma-VAE: Mitigating Shortcut Learning with Generative Classifierscited by 17

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 3; 2026: 1.

4 papers.

  1. 2025Federated Class-Incremental Learning: A Hybrid Approach Using Latent Exemplars and Data-Free Techniques to Address Local and Global Forgettingcited by 9
  2. 2025When Continue Learning Meets Multimodal Large Language Model: A Surveycited by 8
  3. 2026Lifecycle-Aware Federated Continual Learning in Mobile Autonomous Systemscited by 7
  4. 2025Global Pre-fixing, Local Adjusting: A Simple yet Effective Contrastive Strategy for Continual Learningcited by 0