Incremental Few-Shot Learning via Implanting and Compressing
Not on the map. Why?
Abstract (arXiv)
This work focuses on tackling the challenging but realistic visual task of Incremental Few-Shot Learning (IFSL), which requires a model to continually learn novel classes from only a few examples while not forgetting the base classes on which it was pre-trained. Our study reveals that the challenges of IFSL lie in both inter-class separation and novel-class representation. Dur to intra-class variation, a novel class may implicitly leverage the knowledge from multiple base classes to construct its feature representation. Hence, simply reusing the pre-trained embedding space could lead to a scattered feature distribution and result in category confusion. To address such issues, we propose a two-step learning strategy referred to as Implanting and Compressing (IMCO), which optimizes both feature space partition and novel class reconstruction in a systematic manner. Specifically, in the Implanting step, we propose to mimic the data distribution of novel classes with the assistance of data-abundant base set, so that a model could learn semantically-rich features that are beneficial for discriminating between the base and other unseen classes. In the Compressing step, we adapt the feature extractor to precisely represent each novel class for enhancing intra-class compactness, together with a regularized parameter updating rule for preventing aggressive model updating. Finally, we demonstrate that IMCO outperforms competing baselines with a significant margin, both in image classification task and more challenging object detection task.
Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Builds on 34 of its 42 references that are in the map. First posted on arXiv 19 Mar 2022.
Builds on
34 of 42 references.
References · 34
- 2015Deep Residual Learning for Image Recognitioncited by 52,096
- 2017Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networkscited by 4,897
- 2016Overcoming catastrophic forgetting in neural networkscited by 4,787
- 2017Prototypical Networks for Few-shot Learningcited by 3,015
- 2016Learning without Forgettingcited by 2,496
- 2016iCaRL: Incremental Classifier and Representation Learningcited by 2,333
- 2016On Large-Batch Training for Deep Learning: Generalization Gap and Sharp Minimacited by 1,352
- 2017Learning to Compare: Relation Network for Few-Shot Learningcited by 1,143
- 2017PackNet: Adding Multiple Tasks to a Single Network by Iterative Pruningcited by 898
- 2018Riemannian Walk for Incremental Learning: Understanding Forgetting and Intransigencecited by 742
- 2018Overcoming catastrophic forgetting with hard attention to the taskcited by 681
- 2018End-to-End Incremental Learningcited by 597
22 more
- 2015Towards Open Set Deep Networkscited by 538
- 2020Class-incremental learning: survey and performance evaluation on image classificationcited by 449
- 2018Dynamic Few-Shot Visual Learning without Forgettingcited by 385
- 2021Towards Open World Object Detectioncited by 297
- 2019SMART: Robust and Efficient Fine-Tuning for Pre-trained Natural Language Models through Principled Regularized Optimizationcited by 254
- 2017Overcoming Catastrophic Forgetting by Incremental Moment Matchingcited by 243
- 2014Towards Open World Recognitioncited by 226
- 2017Incremental Learning of Object Detectors without Catastrophic Forgettingcited by 208
- 2019Meta R-CNN : Towards General Solver for Instance-level Few-shot Learningcited by 182
- 2021FSCE: Few-Shot Object Detection via Contrastive Proposal Encodingcited by 168
- 2017Generative OpenMax for Multi-Class Open Set Classificationcited by 157
- 2020Better Fine-Tuning by Reducing Representational Collapsecited by 157
- 2020Incremental Few-Shot Object Detectioncited by 102
- 2019Analyzing and Improving Representations with the Soft Nearest Neighbor Losscited by 95
- 2021Beyond Max-Margin: Class Margin Equilibrium for Few-shot Object Detectioncited by 80
- 2021Hallucination Improves Few-Shot Object Detectioncited by 50
- 2021Incremental Few-Shot Instance Segmentationcited by 49
- 2020Distance-Based Regularisation of Deep Networks for Fine-Tuningcited by 48
- 2021Few-Shot Object Detection via Association and DIscriminationcited by 39
- 2021Generalized and Incremental Few-Shot Learning by Explicit Learning and Calibration without Forgettingcited by 33
- 2020One-Shot Image Classification by Learning to Restore Prototypescited by 14
- 2018Gradient-Based Meta-Learning with Learned Layerwise Metric and Subspacecited by 4
Built on by
Citing papers per year: 2022: 2.
2 papers.