mlsanityData to 7 Oct 2026

Few-Shot Lifelong Learning

arXiv 2103.00991 ↗First posted 1 Mar 2021cs.CVcited by 58

Not on the map. Why?

Abstract (arXiv)

Many real-world classification problems often have classes with very few labeled training samples. Moreover, all possible classes may not be initially available for training, and may be given incrementally. Deep learning models need to deal with this two-fold problem in order to perform well in real-life situations. In this paper, we propose a novel Few-Shot Lifelong Learning (FSLL) method that enables deep learning models to perform lifelong/continual learning on few-shot data. Our method selects very few parameters from the model for training every new set of classes instead of training the full model. This helps in preventing overfitting. We choose the few parameters from the model in such a way that only the currently unimportant parameters get selected. By keeping the important parameters in the model intact, our approach minimizes catastrophic forgetting. Furthermore, we minimize the cosine similarity between the new and the old class prototypes in order to maximize their separation, thereby improving the classification performance. We also show that integrating our method with self-supervision improves the model performance significantly. We experimentally show that our method significantly outperforms existing methods on the miniImageNet, CIFAR-100, and CUB-200 datasets. Specifically, we outperform the state-of-the-art method by an absolute margin of 19.27% for the CUB dataset.

Key facts. Cited by 58 papers in the map (data through 7 Oct 2026). Builds on 37 of its 51 references that are in the map. First posted on arXiv 1 Mar 2021.

Builds on

37 of 51 references.

References · 37

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 11,979
  3. 2015Distilling the Knowledge in a Neural Networkcited by 10,076
  4. 2019Momentum Contrast for Unsupervised Visual Representation Learningcited by 7,482
  5. 2017Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networkscited by 4,861
  6. 2016Overcoming catastrophic forgetting in neural networkscited by 4,718
  7. 2017Prototypical Networks for Few-shot Learningcited by 3,001
  8. 2016Learning without Forgettingcited by 2,465
  9. 2016iCaRL: Incremental Classifier and Representation Learningcited by 2,313
  10. 2016Matching Networks for One Shot Learningcited by 2,246
  11. 2015Learning both Weights and Connections for Efficient Neural Networkscited by 1,998
  12. 2017Gradient Episodic Memory for Continual Learningcited by 1,871
25 more
  1. 2018Unsupervised Representation Learning by Predicting Image Rotationscited by 1,586
  2. 2016Context Encoders: Feature Learning by Inpaintingcited by 1,424
  3. 2016Unsupervised Learning of Visual Representations by Solving Jigsaw Puzzlescited by 1,379
  4. 2019Contrastive Multiview Codingcited by 1,297
  5. 2017Continual Learning with Deep Generative Replaycited by 1,152
  6. 2017Learning to Compare: Relation Network for Few-Shot Learningcited by 1,139
  7. 2016Colorful Image Colorizationcited by 1,113
  8. 2017Memory Aware Synapses: Learning what (not) to forgetcited by 1,094
  9. 2015Unsupervised Visual Representation Learning by Context Predictioncited by 1,062
  10. 2018Efficient Lifelong Learning with A-GEMcited by 1,044
  11. 2017PackNet: Adding Multiple Tasks to a Single Network by Iterative Pruningcited by 882
  12. 2019Large Scale Incremental Learningcited by 727
  13. 2017Lifelong Learning with Dynamically Expandable Networkscited by 689
  14. 2018Overcoming catastrophic forgetting with hard attention to the taskcited by 672
  15. 2018End-to-End Incremental Learningcited by 596
  16. 2018Dynamic Few-Shot Visual Learning without Forgettingcited by 385
  17. 2018Meta-Transfer Learning for Few-Shot Learningcited by 306
  18. 2016Learning Representations for Automatic Colorizationcited by 251
  19. 2020Few-Shot Class-Incremental Learningcited by 249
  20. 2017Overcoming Catastrophic Forgetting by Incremental Moment Matchingcited by 243
  21. 2018Reinforced Continual Learningcited by 186
  22. 2019Boosting Few-Shot Visual Learning with Self-Supervisioncited by 140
  23. 2018Rotate your Networks: Better Weight Consolidation and Less Catastrophic Forgettingcited by 127
  24. 2019Lifelong GAN: Continual Learning for Conditional Image Generationcited by 78
  25. 2018Incremental Few-Shot Learning with Attention Attractor Networkscited by 65

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 4; 2022: 13; 2023: 18; 2024: 16; 2025: 5; 2026: 2.

58 papers.

Sort:

By area:

  1. 2023A Comprehensive Survey of Continual Learning: Theory, Method and Applicationcited by 860
  2. 2022Forward Compatible Few-Shot Class-Incremental Learningcited by 125
  3. 2021Overcoming Catastrophic Forgetting in Incremental Few-Shot Learning by Finding Flat Minimacited by 118
  4. 2022Few-Shot Class-Incremental Learning by Sampling Multi-Phase Taskscited by 87
  5. 2023A Comprehensive Survey of Forgetting in Deep Learning Beyond Continual Learningcited by 66
  6. 2023Learning with Fantasy: Semantic-Aware Virtual Contrastive Constraint for Few-Shot Class-Incremental Learningcited by 55
  7. 2023A Survey on Few-Shot Class-Incremental Learningcited by 48
  8. 2022Margin-Based Few-Shot Class-Incremental Learning with Class-Level Overfitting Mitigationcited by 45
  9. 2022On the Soft-Subnetwork for Few-shot Class Incremental Learningcited by 37
  10. 2024Pre-trained Vision and Language Transformers Are Few-Shot Incremental Learnerscited by 35
  11. 2022Memorizing Complementation Network for Few-Shot Class-Incremental Learningcited by 30
  12. 2022Learning with Recoverable Forgettingcited by 29
38 more
  1. 2023Few-shot Class-incremental Learning for Classification and Object Detection: A Surveycited by 25
  2. 2024OrCo: Towards Better Generalization via Orthogonality and Contrast for Few-Shot Class-Incremental Learningcited by 25
  3. 2023S3C: Self-Supervised Stochastic Classifiers for Few-Shot Class-Incremental Learningcited by 23
  4. 2024Generative Multi-modal Models are Good Class-Incremental Learnerscited by 22
  5. 2023Multimodal Parameter-Efficient Few-Shot Class Incremental Learningcited by 19
  6. 2022Few-shot Class-incremental Learning for 3D Point Cloud Objectscited by 13
  7. 2022Task Agnostic Representation Consolidation: a Self-supervised based Continual Learning Approachcited by 13
  8. 2024Rethinking Few-shot Class-incremental Learning: Learning from Yourselfcited by 13
  9. 2022Learnable Distribution Calibration for Few-Shot Class-Incremental Learningcited by 12
  10. 2023When Meta-Learning Meets Online and Continual Learning: A Surveycited by 12
  11. 2024Learning Prompt with Distribution-Based Feature Replay for Few-Shot Class-Incremental Learningcited by 12
  12. 2023LFS-GAN: Lifelong Few-Shot Image Generationcited by 11
  13. 2023Online Continual Learning for Robust Indoor Object Recognitioncited by 9
  14. 2025Continual Unlearning for Text-to-Image Diffusion Models: A Regularization Perspectivecited by 9
  15. 2021Avoiding Forgetting and Allowing Forward Transfer in Continual Learning via Sparse Networkscited by 7
  16. 2023Semantic-visual Guided Transformer for Few-shot Class-incremental Learningcited by 6
  17. 2024Delta: A Cloud-assisted Data Enrichment Framework for On-Device Continual Learningcited by 6
  18. 2023Continual Learning: Forget-free Winning Subnetworks for Video Representationscited by 5
  19. 2023Forget-free Continual Learning with Soft-Winning SubNetworkscited by 4
  20. 2023Few-Shot Continual Learning via Flat-to-Wide Approachescited by 4
  21. 2024Enhanced Few-Shot Class-Incremental Learning via Ensemble Modelscited by 4
  22. 2024Few-Shot Class Incremental Learning via Robust Transformer Approachcited by 4
  23. 2021Incremental Meta-Learning via Episodic Replay Distillation for Few-Shot Image Recognitioncited by 3
  24. 2022Better Generalized Few-Shot Learning Even Without Base Datacited by 3
  25. 2023Continual HyperTransformer: A Meta-Learner for Continual Few-Shot Learningcited by 3
  26. 2024Foundation Model-Powered 3D Few-Shot Class Incremental Learning via Training-free Adaptorcited by 3
  27. 2024Class Balance Matters to Active Class-Incremental Learningcited by 3
  28. 2025Global Convergence of Continual Learning on Non-IID Datacited by 3
  29. 2023Towards Label-Efficient Incremental Learning: A Surveycited by 2
  30. 2024A Bag of Tricks for Few-Shot Class-Incremental Learningcited by 2
  31. 2025DSS-Prompt: Dynamic-Static Synergistic Prompting for Few-Shot Class-Incremental Learningcited by 2
  32. 2021A Strong Baseline for Semi-Supervised Incremental Few-Shot Learningcited by 1
  33. 2022Memory-Based Label-Text Tuning for Few-Shot Class-Incremental Learningcited by 1
  34. 2022Prototypical quadruplet for few-shot class incremental learningcited by 1
  35. 2024PL-FSCIL: Harnessing the Power of Prompts for Few-Shot Class-Incremental Learningcited by 1
  36. 2024Bias Mitigating Few-Shot Class-Incremental Learningcited by 1
  37. 2024Consistency-Guided Asynchronous Contrastive Tuning for Few-Shot Class-Incremental Tuning of Foundation Modelscited by 1
  38. 2024Unsupervised Few-Shot Continual Learning for Remote Sensing Image Scene Classificationcited by 1

50 most cited of 58 All 58 (JSON)