mlsanityData to 14 Oct 2026

On the importance of cross-task features for class-incremental learning

arXiv 2106.11930 ↗First posted 22 Jun 2021cs.LG cs.CVcited by 18

Not on the map. Why?

Abstract (arXiv)

In class-incremental learning, an agent with limited resources needs to learn a sequence of classification tasks, forming an ever growing classification problem, with the constraint of not being able to access data from previous tasks. The main difference with task-incremental learning, where a task-ID is available at inference time, is that the learner also needs to perform cross-task discrimination, i.e. distinguish between classes that have not been seen together. Approaches to tackle this problem are numerous and mostly make use of an external memory (buffer) of non-negligible size. In this paper, we ablate the learning of cross-task features and study its influence on the performance of basic replay strategies used for class-IL. We also define a new forgetting measure for class-incremental learning, and see that forgetting is not the principal cause of low performance. Our experimental results show that future algorithms for class-incremental learning should not only prevent forgetting, but also aim to improve the quality of the cross-task features, and the knowledge transfer between tasks. This is especially important when tasks contain limited amount of data.

Key facts. Cited by 18 papers in the map (data through 14 Oct 2026). Builds on 32 of its 43 references that are in the map. First posted on arXiv 22 Jun 2021.

Builds on

32 of 43 references.

References · 32

  1. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  2. 2014ImageNet Large Scale Visual Recognition Challengecited by 8,498
  3. 2016Overcoming catastrophic forgetting in neural networkscited by 4,787
  4. 2016Learning without Forgettingcited by 2,496
  5. 2016iCaRL: Incremental Classifier and Representation Learningcited by 2,333
  6. 2017Gradient Episodic Memory for Continual Learningcited by 1,898
  7. 2016Understanding intermediate layers using linear classifier probescited by 1,353
  8. 2017Continual Learning with Deep Generative Replaycited by 1,158
  9. 2017Memory Aware Synapses: Learning what (not) to forgetcited by 1,105
  10. 2018Efficient Lifelong Learning with A-GEMcited by 1,058
  11. 2018Riemannian Walk for Incremental Learning: Understanding Forgetting and Intransigencecited by 742
  12. 2019Large Scale Incremental Learningcited by 732
20 more
  1. 2013An Empirical Investigation of Catastrophic Forgetting in Gradient-Based Neural Networkscited by 722
  2. 2018Overcoming catastrophic forgetting with hard attention to the taskcited by 681
  3. 2018End-to-End Incremental Learningcited by 597
  4. 2019Gradient based sample selection for online continual learningcited by 571
  5. 2020Class-incremental learning: survey and performance evaluation on image classificationcited by 449
  6. 2019Online Continual Learning with Maximally Interfered Retrievalcited by 379
  7. 2019On Tiny Episodic Memories in Continual Learningcited by 307
  8. 2018Learning without Memorizingcited by 250
  9. 2020Semantic Drift Compensation for Class-Incremental Learningcited by 241
  10. 2020Mnemonics Training: Multi-Class Incremental Learning without Forgettingcited by 221
  11. 2019REMIND Your Neural Network to Prevent Catastrophic Forgettingcited by 197
  12. 2019Class-incremental Learning via Deep Model Consolidationcited by 173
  13. 2019Overcoming Catastrophic Forgetting with Unlabeled Data in the Wildcited by 111
  14. 2020Generative Feature Replay For Class-Incremental Learningcited by 109
  15. 2020A Comprehensive Study of Class Incremental Learning Algorithms for Visual Taskscited by 107
  16. 2017Encoder Based Lifelong Learningcited by 102
  17. 2016Less-forgetting Learning in Deep Neural Networkscited by 97
  18. 2018Memory Replay GANs: learning to generate images from new categories without forgettingcited by 77
  19. 2020On Class Orderings for Incremental Learningcited by 10
  20. 2020Automatic Recall Machines: Internal Replay, Continual Learning and the Braincited by 7

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2022: 2; 2023: 3; 2024: 7; 2025: 5; 2026: 1.

18 papers.

Sort:

  1. 2024InfLoRA: Interference-Free Low-Rank Adaptation for Continual Learningcited by 198
  2. 2023RanPAC: Random Projections and Pre-trained Models for Continual Learningcited by 164
  3. 2024Continual Learning and Catastrophic Forgettingcited by 77
  4. 2024Resurrecting Old Classes with New Data for Exemplar-Free Continual Learningcited by 33
  5. 2023Dealing with Cross-Task Class Discrimination in Online Continual Learningcited by 24
  6. 2023A Comprehensive Empirical Evaluation on Online Continual Learningcited by 24
  7. 2024Calibrating Higher-Order Statistics for Few-Shot Class-Incremental Learning with Pre-trained Vision Transformerscited by 9
  8. 2024Integrating Present and Past in Unsupervised Continual Learningcited by 8
  9. 2025EKPC: Elastic Knowledge Preservation and Compensation for Class-Incremental Learningcited by 6
  10. 2025CKAA: Cross-subspace Knowledge Alignment and Aggregation for Robust Continual Learningcited by 6
  11. 2022FUSSL: Fuzzy Uncertain Self Supervised Learningcited by 5
  12. 2022Exemplar-free Continual Learning of Vision Transformers via Gated Class-Attention and Cascaded Feature Drift Compensationcited by 4
6 more
  1. 2024Task Confusion and Catastrophic Forgetting in Class-Incremental Learning: A Mathematical Framework for Discriminative and Generative Modelingscited by 4
  2. 2025Autoencoder-Based Hybrid Replay for Class-Incremental Learningcited by 4
  3. 2024The impact of model size on catastrophic forgetting in Online Continual Learningcited by 2
  4. 2025Contrastive Regularization over LoRA for Multimodal Biomedical Image Incremental Learningcited by 2
  5. 2025Global Pre-fixing, Local Adjusting: A Simple yet Effective Contrastive Strategy for Continual Learningcited by 0
  6. 2026Forgetting, plasticity, and co-observation: a third facet of continual learningcited by 0