Knowledge Adaptation Network for Few-Shot Class-Incremental Learning
Not on the map. Why?
Abstract (arXiv)
Few-shot class-incremental learning (FSCIL) aims to incrementally recognize new classes using a few samples while maintaining the performance on previously learned classes. One of the effective methods to solve this challenge is to construct prototypical evolution classifiers. Despite the advancement achieved by most existing methods, the classifier weights are simply initialized using mean features. Because representations for new classes are weak and biased, we argue such a strategy is suboptimal. In this paper, we tackle this issue from two aspects. Firstly, thanks to the development of foundation models, we employ a foundation model, the CLIP, as the network pedestal to provide a general representation for each class. Secondly, to generate a more reliable and comprehensive instance representation, we propose a Knowledge Adapter (KA) module that summarizes the data-specific knowledge from training data and fuses it into the general representation. Additionally, to tune the knowledge learned from the base classes to the upcoming classes, we propose a mechanism of Incremental Pseudo Episode Learning (IPEL) by simulating the actual FSCIL. Taken together, our proposed method, dubbed as Knowledge Adaptation Network (KANet), achieves competitive performance on a wide range of datasets, including CIFAR100, CUB200, and ImageNet-R.
Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Builds on 43 of its 65 references that are in the map. First posted on arXiv 18 Sep 2024.
Builds on
43 of 65 references.
References · 43
- 2017Attention Is All You Needcited by 59,932
- 2015Deep Residual Learning for Image Recognitioncited by 51,809
- 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
- 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
- 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,160
- 2015Distilling the Knowledge in a Neural Networkcited by 10,076
- 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,160
- 2014ImageNet Large Scale Visual Recognition Challengecited by 8,441
- 2021Emerging Properties in Self-Supervised Vision Transformerscited by 6,893
- 2017Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networkscited by 4,861
- 2016Learning without Forgettingcited by 2,465
- 2016iCaRL: Incremental Classifier and Representation Learningcited by 2,313
31 more
- 2020REALM: Retrieval-Augmented Language Model Pre-Trainingcited by 2,126
- 2020The Many Faces of Robustness: A Critical Analysis of Out-of-Distribution Generalizationcited by 1,842
- 2023Self-RAG: Learning to Retrieve, Generate, and Critique through Self-Reflectioncited by 1,800
- 2021GLM: General Language Model Pretraining with Autoregressive Blank Infillingcited by 1,026
- 2021Learning to Prompt for Continual Learningcited by 938
- 2023REPLUG: Retrieval-Augmented Black-Box Language Modelscited by 786
- 2022DualPrompt: Complementary Prompting for Rehearsal-free Continual Learningcited by 635
- 2018End-to-End Incremental Learningcited by 596
- 2022Tip-Adapter: Training-free Adaption of CLIP for Few-shot Classificationcited by 355
- 2020Few-Shot Class-Incremental Learningcited by 249
- 2022S-Prompts Learning with Pre-trained Transformers: An Occam's Razor for Domain Incremental Learningcited by 246
- 2021Few-Shot Incremental Learning with Continually Evolved Classifierscited by 184
- 2023Prompt, Generate, then Cache: Cascade of Foundation Models makes Strong Few-shot Learnerscited by 152
- 2018Task-Agnostic Meta-Learning for Few-shot Learningcited by 151
- 2024Continual Learning for Large Language Models: A Surveycited by 144
- 2022Forward Compatible Few-Shot Class-Incremental Learningcited by 125
- 2021Overcoming Catastrophic Forgetting in Incremental Few-Shot Learning by Finding Flat Minimacited by 118
- 2021Semantic-aware Knowledge Distillation for Few-Shot Class-Incremental Learningcited by 114
- 2023Neural Collapse Inspired Feature-Classifier Alignment for Few-Shot Class Incremental Learningcited by 112
- 2021Self-Promoted Prototype Refinement for Few-Shot Class-Incremental Learningcited by 95
- 2022Constrained Few-shot Class-incremental Learningcited by 93
- 2022Few-Shot Class-Incremental Learning from an Open-Set Perspectivecited by 83
- 2022Memory Replay with Data Compression for Continual Learningcited by 80
- 2021Meta-Learning with Task-Adaptive Loss Function for Few-Shot Learningcited by 49
- 2022Margin-Based Few-Shot Class-Incremental Learning with Class-Level Overfitting Mitigationcited by 45
- 2023Few-Shot Class-Incremental Learning via Training-Free Prototype Calibrationcited by 44
- 2022Generating Representative Samples for Few-Shot Classificationcited by 39
- 2022On the Soft-Subnetwork for Few-shot Class Incremental Learningcited by 37
- 2022Memorizing Complementation Network for Few-Shot Class-Incremental Learningcited by 30
- 2021Balanced softmax cross-entropy for incremental learning with and without memorycited by 13
- 2022CAD: Co-Adapting Discriminative Features for Improved Few-Shot Classificationcited by 9
Built on by
Citing papers per year: 2025: 1.
1 paper.