mlsanityData to 7 Oct 2026

SRIL: Selective Regularization for Class-Incremental Learning

arXiv 2305.05175 ↗First posted 9 May 2023cs.CVcited by 0

Not on the map. Why?

Abstract (arXiv)

Human intelligence gradually accepts new information and accumulates knowledge throughout the lifespan. However, deep learning models suffer from a catastrophic forgetting phenomenon, where they forget previous knowledge when acquiring new information. Class-Incremental Learning aims to create an integrated model that balances plasticity and stability to overcome this challenge. In this paper, we propose a selective regularization method that accepts new knowledge while maintaining previous knowledge. We first introduce an asymmetric feature distillation method for old and new classes inspired by cognitive science, using the gradient of classification and knowledge distillation losses to determine whether to perform pattern completion or pattern separation. We also propose a method to selectively interpolate the weight of the previous model for a balance between stability and plasticity, and we adjust whether to transfer through model confidence to ensure the performance of the previous class and enable exploratory learning. We validate the effectiveness of the proposed method, which surpasses the performance of existing methods through extensive experimental protocols using CIFAR-100, ImageNet-Subset, and ImageNet-Full.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 32 of its 47 references that are in the map. First posted on arXiv 9 May 2023.

Builds on

32 of 47 references.

References · 32

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  3. 2015Distilling the Knowledge in a Neural Networkcited by 10,076
  4. 2016Overcoming catastrophic forgetting in neural networkscited by 4,718
  5. 2016Learning without Forgettingcited by 2,465
  6. 2016iCaRL: Incremental Classifier and Representation Learningcited by 2,313
  7. 2019Similarity of Neural Network Representations Revisitedcited by 1,695
  8. 2022Model soups: averaging weights of multiple fine-tuned models improves accuracy without increasing inference timecited by 1,366
  9. 2016Progressive Neural Networkscited by 1,332
  10. 2018Averaging Weights Leads to Wider Optima and Better Generalizationcited by 1,194
  11. 2017Memory Aware Synapses: Learning what (not) to forgetcited by 1,094
  12. 2021Robust fine-tuning of zero-shot modelscited by 785
20 more
  1. 2018Riemannian Walk for Incremental Learning: Understanding Forgetting and Intransigencecited by 734
  2. 2019Large Scale Incremental Learningcited by 727
  3. 2013An Empirical Investigation of Catastrophic Forgetting in Gradient-Based Neural Networkscited by 711
  4. 2017Lifelong Learning with Dynamically Expandable Networkscited by 689
  5. 2018End-to-End Incremental Learningcited by 596
  6. 2017No Fuss Distance Metric Learning using Proxiescited by 225
  7. 2020Mnemonics Training: Multi-Class Incremental Learning without Forgettingcited by 220
  8. 2021New Insights on Reducing Abrupt Representation Change in Online Continual Learningcited by 215
  9. 2019Compacting, Picking and Growing for Unforgetting Continual Learningcited by 207
  10. 2020Adaptive Aggregation Networks for Class-Incremental Learningcited by 149
  11. 2021Distilling Causal Effect of Data in Class-Incremental Learningcited by 117
  12. 2022Class-Incremental Learning by Knowledge Distillation with Adaptive Feature Consolidationcited by 113
  13. 2018Adapting Auxiliary Losses Using Gradient Similaritycited by 98
  14. 2021On Learning the Geodesic Path for Incremental Learningcited by 85
  15. 2023RMM: Reinforced Memory Management for Class-Incremental Learningcited by 66
  16. 2020CPR: Classifier-Projection Regularization for Continual Learningcited by 62
  17. 2021Continual Learning for Real-World Autonomous Systems: Algorithms, Challenges and Frameworkscited by 56
  18. 2022Momentum-based Weight Interpolation of Strong Zero-Shot Models for Continual Learningcited by 19
  19. 2022Class-Incremental Learning with Cross-Space Clustering and Controlled Transfercited by 15
  20. 2022Weight Averaging: A Simple Yet Effective Method to Overcome Catastrophic Forgetting in Automatic Speech Recognitioncited by 10

Built on by

Not cited yet.