mlsanityData to 14 Oct 2026

Big-model Driven Few-shot Continual Learning

arXiv 2309.00862 ↗First posted 2 Sep 2023cs.CVcited by 0

Not on the map. Why?

Abstract (arXiv)

Few-shot continual learning (FSCL) has attracted intensive attention and achieved some advances in recent years, but now it is difficult to again make a big stride in accuracy due to the limitation of only few-shot incremental samples. Inspired by distinctive human cognition ability in life learning, in this work, we propose a novel Big-model driven Few-shot Continual Learning (B-FSCL) framework to gradually evolve the model under the traction of the world's big-models (like human accumulative knowledge). Specifically, we perform the big-model driven transfer learning to leverage the powerful encoding capability of these existing big-models, which can adapt the continual model to a few of newly added samples while avoiding the over-fitting problem. Considering that the big-model and the continual model may have different perceived results for the identical images, we introduce an instance-level adaptive decision mechanism to provide the high-level flexibility cognitive support adjusted to varying samples. In turn, the adaptive decision can be further adopted to optimize the parameters of the continual model, performing the adaptive distillation of big-model's knowledge information. Experimental results of our proposed B-FSCL on three popular datasets (including CIFAR100, minilmageNet and CUB200) completely surpass all state-of-the-art FSCL methods.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 27 of its 40 references that are in the map. First posted on arXiv 2 Sep 2023.

Builds on

27 of 40 references.

References · 27

  1. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  2. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  3. 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,364
  4. 2014ImageNet Large Scale Visual Recognition Challengecited by 8,498
  5. 2023Segment Anythingcited by 8,234
  6. 2021Scaling Up Visual and Vision-Language Representation Learning With Noisy Text Supervisioncited by 3,864
  7. 2023Grounding DINO: Marrying DINO with Grounded Pre-Training for Open-Set Object Detectioncited by 3,332
  8. 2016iCaRL: Incremental Classifier and Representation Learningcited by 2,333
  9. 2016Matching Networks for One Shot Learningcited by 2,255
  10. 2021CLIP-Adapter: Better Vision-Language Models with Feature Adapterscited by 1,026
  11. 2018End-to-End Incremental Learningcited by 597
  12. 2020Few-Shot Class-Incremental Learningcited by 251
15 more
  1. 2019Meta-Learning Representations for Continual Learningcited by 189
  2. 2021Few-Shot Incremental Learning with Continually Evolved Classifierscited by 187
  3. 2020Negative Margin Matters: Understanding Margin in Few-shot Classificationcited by 129
  4. 2022Forward Compatible Few-Shot Class-Incremental Learningcited by 127
  5. 2023Neural Collapse Inspired Feature-Classifier Alignment for Few-Shot Class Incremental Learningcited by 116
  6. 2021Semantic-aware Knowledge Distillation for Few-Shot Class-Incremental Learningcited by 115
  7. 2023Preventing Zero-Shot Transfer Degradation in Continual Learning of Vision-Language Modelscited by 112
  8. 2021Self-Promoted Prototype Refinement for Few-Shot Class-Incremental Learningcited by 96
  9. 2022Constrained Few-shot Class-incremental Learningcited by 94
  10. 2022Few-Shot Class-Incremental Learning from an Open-Set Perspectivecited by 84
  11. 2022CLIP model is an Efficient Continual Learnercited by 65
  12. 2022Don't Stop Learning: Towards Continual Learning for the CLIP Modelcited by 61
  13. 2021Few-Shot Lifelong Learningcited by 59
  14. 2023CLIP-guided Prototype Modulating for Few-shot Action Recognitioncited by 41
  15. 2023S3C: Self-Supervised Stochastic Classifiers for Few-Shot Class-Incremental Learningcited by 23

Built on by

Not cited yet.