mlsanityData to 7 Oct 2026

Few Shot Class Incremental Learning using Vision-Language models

arXiv 2405.01040 ↗First posted 2 May 2024cs.CV cs.CL eess.IVcited by 0

On the map: Multimodal LLMs / vision-language models → Prompt learning and adaptation of CLIP

Abstract (arXiv)

Recent advancements in deep learning have demonstrated remarkable performance comparable to human capabilities across various supervised computer vision tasks. However, the prevalent assumption of having an extensive pool of training data encompassing all classes prior to model training often diverges from real-world scenarios, where limited data availability for novel classes is the norm. The challenge emerges in seamlessly integrating new classes with few samples into the training data, demanding the model to adeptly accommodate these additions without compromising its performance on base classes. To address this exigency, the research community has introduced several solutions under the realm of few-shot class incremental learning (FSCIL). In this study, we introduce an innovative FSCIL framework that utilizes language regularizer and subspace regularizer. During base training, the language regularizer helps incorporate semantic information extracted from a Vision-Language model. The subspace regularizer helps in facilitating the model's acquisition of nuanced connections between image and text semantics inherent to base classes during incremental training. Our proposed framework not only empowers the model to embrace novel classes with limited data, but also ensures the preservation of performance on base classes. To substantiate the efficacy of our approach, we conduct comprehensive experiments on three distinct FSCIL benchmarks, where our framework attains state-of-the-art performance.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Prompt learning and adaptation of CLIP, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 24 of its 37 references that are in the map. First posted on arXiv 2 May 2024.

Builds on

24 of 37 references.

References · 24

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  2. 2022BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and Generationcited by 4,310
  3. 2016iCaRL: Incremental Classifier and Representation Learningcited by 2,313
  4. 2016Enriching Word Vectors with Subword Informationcited by 1,578
  5. 2018End-to-End Incremental Learningcited by 596
  6. 2020Class-incremental learning: survey and performance evaluation on image classificationcited by 446
  7. 2018Dynamic Few-Shot Visual Learning without Forgettingcited by 385
  8. 2022Scaling Language-Image Pre-training via Maskingcited by 317
  9. 2020Few-Shot Class-Incremental Learningcited by 249
  10. 2023Class-Incremental Learning: A Surveycited by 209
  11. 2023A Simple Framework for Open-Vocabulary Segmentation and Detectioncited by 197
  12. 2021Few-Shot Incremental Learning with Continually Evolved Classifierscited by 184
12 more
  1. 2020A Review of Generalized Zero-Shot Learning Methodscited by 159
  2. 2021Overcoming Catastrophic Forgetting in Incremental Few-Shot Learning by Finding Flat Minimacited by 118
  3. 2021Semantic-aware Knowledge Distillation for Few-Shot Class-Incremental Learningcited by 114
  4. 2020Generative Feature Replay For Class-Incremental Learningcited by 109
  5. 2021Self-Promoted Prototype Refinement for Few-Shot Class-Incremental Learningcited by 95
  6. 2022Few-Shot Class-Incremental Learning via Entropy-Regularized Data-Free Replaycited by 70
  7. 2018Incremental Few-Shot Learning with Attention Attractor Networkscited by 65
  8. 2021Few-Shot Lifelong Learningcited by 58
  9. 2021Class-Incremental Learning with Generative Classifierscited by 39
  10. 2020XtarNet: Learning to Extract Task-Adaptive Representation for Incremental Few-Shot Learningcited by 30
  11. 2019Baby steps towards few-shot learning with multiple semanticscited by 29
  12. 2021Learnable Expansion-and-Compression Network for Few-shot Class-Incremental Learningcited by 11

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2024A streamlined Approach to Multimodal Few-Shot Class Incremental Learning for Fine-Grained Datasets
  2. 2026Feature-Space Generative Models for One-Shot Class-Incremental Learning
  3. 2026Learning to Curate What You Generate for Generalizable Few-Shot Class-Incremental Learning
  4. 2026PRiSM: Prototype Regularization for Few-Shot VLMs
  5. 2024Making Large Vision Language Models to be Good Few-shot Learners
  6. 2023FILP-3D: Enhancing 3D Few-shot Class-incremental Learning with Pre-trained Vision-Language Models
  7. 2024Few-Shot Recognition via Stage-Wise Retrieval-Augmented Finetuning
  8. 2022Exploiting Category Names for Few-Shot Classification with Vision-Language Models