mlsanityData to 7 Oct 2026

On the Soft-Subnetwork for Few-shot Class Incremental Learning

arXiv 2209.07529 ↗First posted 15 Sep 2022cs.LG cs.AIcited by 37

Not on the map. Why?

Abstract (arXiv)

Inspired by Regularized Lottery Ticket Hypothesis (RLTH), which hypothesizes that there exist smooth (non-binary) subnetworks within a dense network that achieve the competitive performance of the dense network, we propose a few-shot class incremental learning (FSCIL) method referred to as Soft-SubNetworks (SoftNet). Our objective is to learn a sequence of sessions incrementally, where each session only includes a few training instances per class while preserving the knowledge of the previously learned ones. SoftNet jointly learns the model weights and adaptive non-binary soft masks at a base training session in which each mask consists of the major and minor subnetwork; the former aims to minimize catastrophic forgetting during training, and the latter aims to avoid overfitting to a few samples in each new training session. We provide comprehensive empirical validations demonstrating that our SoftNet effectively tackles the few-shot incremental learning problem by surpassing the performance of state-of-the-art baselines over benchmark datasets.

Key facts. Cited by 37 papers in the map (data through 7 Oct 2026). Builds on 48 of its 69 references that are in the map. First posted on arXiv 15 Sep 2022.

Builds on

48 of 69 references.

References · 48

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2016Learning without Forgettingcited by 2,465
  3. 2016iCaRL: Incremental Classifier and Representation Learningcited by 2,313
  4. 2018The Lottery Ticket Hypothesis: Finding Sparse, Trainable Neural Networkscited by 2,250
  5. 2013Estimating or Propagating Gradients Through Stochastic Neurons for Conditional Computationcited by 2,031
  6. 2016Progressive Neural Networkscited by 1,332
  7. 2016Optimization Methods for Large-Scale Machine Learningcited by 1,140
  8. 2017PackNet: Adding Multiple Tasks to a Single Network by Iterative Pruningcited by 882
  9. 2019Decoupling Representation and Classifier for Long-Tailed Recognitioncited by 806
  10. 2018Riemannian Walk for Incremental Learning: Understanding Forgetting and Intransigencecited by 734
  11. 2019Large Scale Incremental Learningcited by 727
  12. 2013An Empirical Investigation of Catastrophic Forgetting in Gradient-Based Neural Networkscited by 711
36 more
  1. 2017Lifelong Learning with Dynamically Expandable Networkscited by 689
  2. 2018Overcoming catastrophic forgetting with hard attention to the taskcited by 672
  3. 2018End-to-End Incremental Learningcited by 596
  4. 2019Fantastic Generalization Measures and Where to Find Themcited by 507
  5. 2019Learn to Grow: A Continual Structure Learning Framework for Overcoming Catastrophic Forgettingcited by 302
  6. 2019PyHessian: Neural Networks Through the Lens of the Hessiancited by 291
  7. 2020Few-Shot Class-Incremental Learningcited by 249
  8. 2019What's Hidden in a Randomly Weighted Neural Network?cited by 236
  9. 2020Supermasks in Superpositioncited by 223
  10. 2018Task-Free Continual Learningcited by 219
  11. 2019Deconstructing Lottery Tickets: Zeros, Signs, and the Supermaskcited by 206
  12. 2018Reinforced Continual Learningcited by 186
  13. 2021Few-Shot Incremental Learning with Continually Evolved Classifierscited by 184
  14. 2017FearNet: Brain-Inspired Model for Incremental Learningcited by 177
  15. 2020Adaptive Aggregation Networks for Class-Incremental Learningcited by 149
  16. 2019Scalable and Order-robust Continual Learning with Additive Parameter Decompositioncited by 125
  17. 2021Overcoming Catastrophic Forgetting in Incremental Few-Shot Learning by Finding Flat Minimacited by 118
  18. 2021Semantic-aware Knowledge Distillation for Few-Shot Class-Incremental Learningcited by 114
  19. 2021Online Coreset Selection for Rehearsal-based Continual Learningcited by 114
  20. 2018Alleviating catastrophic forgetting using context-dependent gating and synaptic stabilizationcited by 104
  21. 2021Self-Promoted Prototype Refinement for Few-Shot Class-Incremental Learningcited by 95
  22. 2022Constrained Few-shot Class-incremental Learningcited by 93
  23. 2022Few-Shot Class-Incremental Learning by Sampling Multi-Phase Taskscited by 87
  24. 2022Few-Shot Class-Incremental Learning from an Open-Set Perspectivecited by 83
  25. 2020Good Subnetworks Provably Exist: Pruning via Greedy Forward Selectioncited by 70
  26. 2022Few-Shot Class-Incremental Learning via Entropy-Regularized Data-Free Replaycited by 70
  27. 2018Incremental Few-Shot Learning with Attention Attractor Networkscited by 65
  28. 2020Gradient-based Editing of Memory Examples for Online Task-free Continual Learningcited by 62
  29. 2021Few-Shot Lifelong Learningcited by 58
  30. 2019Discovering Neural Wiringscited by 56
  31. 2021Subspace Regularizers for Few-Shot Class Incremental Learningcited by 54
  32. 2019Task Agnostic Continual Learning via Meta Learningcited by 53
  33. 2019Continuous Meta-Learning without Taskscited by 50
  34. 2022Continual Normalization: Rethinking Batch Normalization for Online Continual Learningcited by 40
  35. 2019A comprehensive, application-oriented study of catastrophic forgetting in DNNscited by 35
  36. 2022Meta-ticket: Finding optimal subnetworks for few-shot learning within randomly initialized neural networkscited by 4

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 9; 2024: 16; 2025: 11; 2026: 1.

37 papers.

Sort:

By area:

  1. 2023A Survey on Few-Shot Class-Incremental Learningcited by 48
  2. 2024Compositional Few-Shot Class-Incremental Learningcited by 18
  3. 2023Carpe Diem: On the Evaluation of World Knowledge in Lifelong Language Modelscited by 17
  4. 2023Non-Exemplar Online Class-incremental Continual Learning via Dual-prototype Self-augment and Refinementcited by 13
  5. 2024Rethinking Few-shot Class-incremental Learning: Learning from Yourselfcited by 13
  6. 2024Calibrating Higher-Order Statistics for Few-Shot Class-Incremental Learning with Pre-trained Vision Transformerscited by 9
  7. 2025Revisiting Pool-based Prompt Learning for Few-shot Class-incremental Learningcited by 8
  8. 2024NTK-Guided Few-Shot Class Incremental Learningcited by 7
  9. 2023Semantic-visual Guided Transformer for Few-shot Class-incremental Learningcited by 6
  10. 2023Continual Learning: Forget-free Winning Subnetworks for Video Representationscited by 5
  11. 2023Forget-free Continual Learning with Soft-Winning SubNetworkscited by 4
  12. 2023Progressive Fourier Neural Representation for Sequential Video Compilationcited by 4
25 more
  1. 2023HyperMask: Adaptive Hypernetwork-based Masks for Continual Learningcited by 4
  2. 2024Self-Regulated Neurogenesis for Online Data-Incremental Learningcited by 3
  3. 2025Latest Advancements Towards Catastrophic Forgetting under Data Scarcity: A Comprehensive Survey on Few-Shot Class Incremental Learningcited by 3
  4. 2025Learn by Reasoning: Analogical Weight Generation for Few-Shot Class-Incremental Learningcited by 3
  5. 2025Mitigating Catastrophic Forgetting in the Incremental Learning of Medical Imagescited by 3
  6. 2024A Bag of Tricks for Few-Shot Class-Incremental Learningcited by 2
  7. 2024I2CANSAY:Inter-Class Analogical Augmentation and Intra-Class Significance Analysis for Non-Exemplar Online Task-Free Continual Learningcited by 2
  8. 2024Covariance-based Space Regularization for Few-shot Class Incremental Learningcited by 2
  9. 2025Tripartite Weight-Space Ensemble for Few-Shot Class-Incremental Learningcited by 2
  10. 2024Bias Mitigating Few-Shot Class-Incremental Learningcited by 1
  11. 2024Controllable Relation Disentanglement for Few-Shot Class-Incremental Learningcited by 1
  12. 2024Consistency-Guided Asynchronous Contrastive Tuning for Few-Shot Class-Incremental Tuning of Foundation Modelscited by 1
  13. 2024Unsupervised Few-Shot Continual Learning for Remote Sensing Image Scene Classificationcited by 1
  14. 2024Continual Deep Learning on the Edge via Stochastic Local Competition among Subnetworkscited by 1
  15. 2024Knowledge Adaptation Network for Few-Shot Class-Incremental Learningcited by 1
  16. 2025Does Prior Data Matter? Exploring Joint Training in the Context of Few-Shot Class-Incremental Learningcited by 1
  17. 2023Knowledge Transfer-Driven Few-Shot Class-Incremental Learningcited by 0
  18. 2024AggSS: An Aggregated Self-Supervised Approach for Class-Incremental Learningcited by 0
  19. 2024Soft-TransFormers for Continual Learningcited by 0
  20. 2025Unlocking [CLS] Features for Continual Post-Trainingcited by 0
  21. 2025MEGA: Second-Order Gradient Alignment for Catastrophic Forgetting Mitigation in GFSCILcited by 0
  22. 2025Diffusion-Classifier Synergy: Reward-Aligned Learning via Mutual Boosting Loop for FSCILcited by 0
  23. 2025SAFA-SNN: Sparsity-Aware On-Device Few-Shot Class-Incremental Learning with Fast-Adaptive Structure of Spiking Neural Networkcited by 0
  24. 2025Breaking Forgetting: Training-Free Few-Shot Class-Incremental Learning via Conditional Diffusioncited by 0
  25. 2026CD^2: Constrained Dataset Distillation for Few-Shot Class-Incremental Learningcited by 0