mlsanityData to 14 Oct 2026

Learnability and Algorithm for Continual Learning

arXiv 2306.12646 ↗First posted 22 Jun 2023cs.LG cs.CVcited by 28

Not on the map. Why?

Abstract (arXiv)

This paper studies the challenging continual learning (CL) setting of Class Incremental Learning (CIL). CIL learns a sequence of tasks consisting of disjoint sets of concepts or classes. At any time, a single model is built that can be applied to predict/classify test instances of any classes learned thus far without providing any task related information for each test instance. Although many techniques have been proposed for CIL, they are mostly empirical. It has been shown recently that a strong CIL system needs a strong within-task prediction (WP) and a strong out-of-distribution (OOD) detection for each task. However, it is still not known whether CIL is actually learnable. This paper shows that CIL is learnable. Based on the theory, a new CIL algorithm is also proposed. Experimental results demonstrate its effectiveness.

Key facts. Cited by 28 papers in the map (data through 14 Oct 2026). Builds on 65 of its 77 references that are in the map. First posted on arXiv 22 Jun 2023.

Builds on

65 of 77 references.

References · 65

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  2. 2016Overcoming catastrophic forgetting in neural networkscited by 4,787
  3. 2020Training data-efficient image transformers & distillation through attentioncited by 4,586
  4. 2019Parameter-Efficient Transfer Learning for NLPcited by 4,236
  5. 2016Learning without Forgettingcited by 2,496
  6. 2016A Baseline for Detecting Misclassified and Out-of-Distribution Examples in Neural Networkscited by 2,385
  7. 2016iCaRL: Incremental Classifier and Representation Learningcited by 2,333
  8. 2017Gradient Episodic Memory for Continual Learningcited by 1,898
  9. 2016Progressive Neural Networkscited by 1,361
  10. 2017Continual Learning with Deep Generative Replaycited by 1,158
  11. 2018Experience Replay for Continual Learningcited by 1,095
  12. 2018Efficient Lifelong Learning with A-GEMcited by 1,058
38 more
  1. 2021Learning to Prompt for Continual Learningcited by 944
  2. 2020Dark Experience for General Continual Learning: a Strong, Simple Baselinecited by 911
  3. 2017PackNet: Adding Multiple Tasks to a Single Network by Iterative Pruningcited by 898
  4. 2019Large Scale Incremental Learningcited by 732
  5. 2018Overcoming catastrophic forgetting with hard attention to the taskcited by 681
  6. 2019Three scenarios for continual learningcited by 614
  7. 2018End-to-End Incremental Learningcited by 597
  8. 2018Progress & Compress: A scalable framework for continual learningcited by 520
  9. 2022ViM: Out-Of-Distribution with Virtual-logit Matchingcited by 405
  10. 2019Online Continual Learning with Maximally Interfered Retrievalcited by 379
  11. 2016Expert Gate: Lifelong Learning with a Network of Expertscited by 372
  12. 2021DER: Dynamically Expandable Representation for Class Incremental Learningcited by 355
  13. 2019On Tiny Episodic Memories in Continual Learningcited by 307
  14. 2019Continual learning with hypernetworkscited by 273
  15. 2018Learning without Memorizingcited by 250
  16. 2021Co2L: Contrastive Continual Learningcited by 244
  17. 2020Supermasks in Superpositioncited by 227
  18. 2020Mnemonics Training: Multi-Class Incremental Learning without Forgettingcited by 221
  19. 2019Compacting, Picking and Growing for Unforgetting Continual Learningcited by 207
  20. 2018Online Structured Laplace Approximations For Overcoming Catastrophic Forgettingcited by 201
  21. 2018Reinforced Continual Learningcited by 186
  22. 2019Learning to Remember: A Synaptic Plasticity Driven Framework for Continual Learningcited by 182
  23. 2018Continual Learning of Context-dependent Processing in Neural Networkscited by 179
  24. 2017FearNet: Brain-Inspired Model for Incremental Learningcited by 177
  25. 2020Using Hindsight to Anchor Past Knowledge in Continual Learningcited by 161
  26. 2020Adaptive Aggregation Networks for Class-Incremental Learningcited by 149
  27. 2019Uncertainty-based Continual Learning with Adaptive Regularizationcited by 146
  28. 2020Conditional Channel Gated Networks for Task-Aware Continual Learningcited by 115
  29. 2019Overcoming Catastrophic Forgetting with Unlabeled Data in the Wildcited by 111
  30. 2022Is Out-of-Distribution Detection Learnable?cited by 109
  31. 2020Continual Learning in Low-rank Orthogonal Subspacescited by 99
  32. 2019Lifelong Machine Learning with Deep Streaming Linear Discriminant Analysiscited by 92
  33. 2021Achieving Forgetting Prevention and Knowledge Transfer in Continual Learningcited by 91
  34. 2020iTAML: An Incremental Task-Agnostic Meta-learning Approachcited by 90
  35. 2021Continual Learning in the Teacher-Student Setup: Impact of Task Similaritycited by 88
  36. 2022Memory Replay with Data Compression for Continual Learningcited by 82
  37. 2021Continual Learning of a Mixed Sequence of Similar and Dissimilar Taskscited by 81
  38. 2022A Theoretical Study on Solving Continual Learningcited by 80

50 most cited of 65 All 65 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 5; 2024: 7; 2025: 14; 2026: 2.

28 papers.

Sort:

By area:

  1. 2023A Unified Approach to Domain Incremental Learning with Memory: Theory and Algorithmcited by 33
  2. 2025HiDe-LLaVA: Hierarchical Decoupling for Continual Instruction Tuning of Multimodal Large Language Modelcited by 31
  3. 2023Class Incremental Learning via Likelihood Ratio Based Task Predictioncited by 19
  4. 2024PILoRA: Prototype Guided Incremental LoRA for Federated Class-Incremental Learningcited by 19
  5. 2023Class Incremental Learning with Self-Supervised Pre-Training and Prototype Learningcited by 18
  6. 2025Out-of-Distribution Generalization in Time Series: A Surveycited by 16
  7. 2025Addressing Imbalanced Domain-Incremental Learning through Dual-Balance Collaborative Expertscited by 13
  8. 2025MCITlib: Multimodal Continual Instruction Tuning Library and Benchmarkcited by 13
  9. 2024Replay-and-Forget-Free Graph Class-Incremental Learning: A Task Profiling and Prompting Approachcited by 12
  10. 2025Federated Continual Instruction Tuningcited by 12
  11. 2023Continual Evidential Deep Learning for Out-of-Distribution Detectioncited by 11
  12. 2024PASS++: A Dual Bias Reduction Framework for Non-Exemplar Class-Incremental Learningcited by 8
16 more
  1. 2025On the Discrimination and Consistency for Exemplar-Free Class Incremental Learningcited by 8
  2. 2023Sub-network Discovery and Soft-masking for Continual Learning of Mixed Taskscited by 7
  3. 2024In-context Continual Learning Assisted by an External Continual Learnercited by 7
  4. 2025AnaCP: Toward Upper-Bound Continual Learning via Analytic Contrastive Projectioncited by 7
  5. 2024Continual Learning Using Only Large Language Model Promptingcited by 5
  6. 2025Active Learning for Continual Learning: Keeping the Past Alive in the Presentcited by 5
  7. 2025Continual Hyperbolic Learning of Instances and Classescited by 5
  8. 2024DESIRE: Dynamic Knowledge Consolidation for Rehearsal-Free Continual Learningcited by 4
  9. 2025Achieving Upper Bound Accuracy of Joint Training in Continual Learningcited by 4
  10. 2025Global Convergence of Continual Learning on Non-IID Datacited by 3
  11. 2025Humans Coexist, So Must Embodied Artificial Agentscited by 2
  12. 2025Buffer-free Class-Incremental Learning with Out-of-Distribution Detectioncited by 2
  13. 2026CL-VISTA: Benchmarking Continual Learning in Video Large Language Modelscited by 2
  14. 2024Balancing the Causal Effects in Class-Incremental Learningcited by 1
  15. 2025On Understanding of the Dynamics of Model Capacity in Continual Learningcited by 1
  16. 2026Exploring the Impact of Parameter Update Magnitude on Forgetting and Generalization of Continual Learningcited by 1