# Few Shot Class Incremental Learning using Vision-Language models

arXiv: https://arxiv.org/abs/2405.01040 · HTML: https://mlsanity.com/paper/2405.01040/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in Prompt learning and adaptation of CLIP, one of 16 directions in Multimodal LLMs / vision-language models.
- Builds on 24 of its 37 references that are in the map.
- First posted on arXiv 2 May 2024.

## Placed in

- Multimodal LLMs / vision-language models / [Prompt learning and adaptation of CLIP](https://mlsanity.com/mllm/prompt-learning-and-adaptation-of-clip/)

## Builds on (most cited first, 24 of 24)

- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and Generation](https://mlsanity.com/paper/2201.12086/) (2022)
- [iCaRL: Incremental Classifier and Representation Learning](https://mlsanity.com/paper/1611.07725/) (2016)
- [Enriching Word Vectors with Subword Information](https://mlsanity.com/paper/1607.04606/) (2016)
- [End-to-End Incremental Learning](https://mlsanity.com/paper/1807.09536/) (2018)
- [Class-incremental learning: survey and performance evaluation on image classification](https://mlsanity.com/paper/2010.15277/) (2020)
- [Dynamic Few-Shot Visual Learning without Forgetting](https://mlsanity.com/paper/1804.09458/) (2018)
- [Scaling Language-Image Pre-training via Masking](https://mlsanity.com/paper/2212.00794/) (2022)
- [Few-Shot Class-Incremental Learning](https://mlsanity.com/paper/2004.10956/) (2020)
- [Class-Incremental Learning: A Survey](https://mlsanity.com/paper/2302.03648/) (2023)
- [A Simple Framework for Open-Vocabulary Segmentation and Detection](https://mlsanity.com/paper/2303.08131/) (2023)
- [Few-Shot Incremental Learning with Continually Evolved Classifiers](https://mlsanity.com/paper/2104.03047/) (2021)
- [A Review of Generalized Zero-Shot Learning Methods](https://mlsanity.com/paper/2011.08641/) (2020)
- [Overcoming Catastrophic Forgetting in Incremental Few-Shot Learning by Finding Flat Minima](https://mlsanity.com/paper/2111.01549/) (2021)
- [Semantic-aware Knowledge Distillation for Few-Shot Class-Incremental Learning](https://mlsanity.com/paper/2103.04059/) (2021)
- [Generative Feature Replay For Class-Incremental Learning](https://mlsanity.com/paper/2004.09199/) (2020)
- [Self-Promoted Prototype Refinement for Few-Shot Class-Incremental Learning](https://mlsanity.com/paper/2107.08918/) (2021)
- [Few-Shot Class-Incremental Learning via Entropy-Regularized Data-Free Replay](https://mlsanity.com/paper/2207.11213/) (2022)
- [Incremental Few-Shot Learning with Attention Attractor Networks](https://mlsanity.com/paper/1810.07218/) (2018)
- [Few-Shot Lifelong Learning](https://mlsanity.com/paper/2103.00991/) (2021)
- [Class-Incremental Learning with Generative Classifiers](https://mlsanity.com/paper/2104.10093/) (2021)
- [XtarNet: Learning to Extract Task-Adaptive Representation for Incremental Few-Shot Learning](https://mlsanity.com/paper/2003.08561/) (2020)
- [Baby steps towards few-shot learning with multiple semantics](https://mlsanity.com/paper/1906.01905/) (2019)
- [Learnable Expansion-and-Compression Network for Few-shot Class-Incremental Learning](https://mlsanity.com/paper/2104.02281/) (2021)

## Abstract (arXiv)

Recent advancements in deep learning have demonstrated remarkable performance comparable to human capabilities across various supervised computer vision tasks. However, the prevalent assumption of having an extensive pool of training data encompassing all classes prior to model training often diverges from real-world scenarios, where limited data availability for novel classes is the norm. The challenge emerges in seamlessly integrating new classes with few samples into the training data, demanding the model to adeptly accommodate these additions without compromising its performance on base classes. To address this exigency, the research community has introduced several solutions under the realm of few-shot class incremental learning (FSCIL). In this study, we introduce an innovative FSCIL framework that utilizes language regularizer and subspace regularizer. During base training, the language regularizer helps incorporate semantic information extracted from a Vision-Language model. The subspace regularizer helps in facilitating the model's acquisition of nuanced connections between image and text semantics inherent to base classes during incremental training. Our proposed framework not only empowers the model to embrace novel classes with limited data, but also ensures the preservation of performance on base classes. To substantiate the efficacy of our approach, we conduct comprehensive experiments on three distinct FSCIL benchmarks, where our framework attains state-of-the-art performance.