mlsanityData to 7 Oct 2026

Few-Shot Class Incremental Learning with Attention-Aware Self-Adaptive Prompt

arXiv 2403.09857 ↗First posted 14 Mar 2024cs.LG cs.AI cs.CVcited by 15

Not on the map. Why?

Abstract (arXiv)

Few-Shot Class-Incremental Learning (FSCIL) models aim to incrementally learn new classes with scarce samples while preserving knowledge of old ones. Existing FSCIL methods usually fine-tune the entire backbone, leading to overfitting and hindering the potential to learn new classes. On the other hand, recent prompt-based CIL approaches alleviate forgetting by training prompts with sufficient data in each task. In this work, we propose a novel framework named Attention-aware Self-adaptive Prompt (ASP). ASP encourages task-invariant prompts to capture shared knowledge by reducing specific information from the attention aspect. Additionally, self-adaptive task-specific prompts in ASP provide specific information and transfer knowledge from old classes to new classes with an Information Bottleneck learning objective. In summary, ASP prevents overfitting on base task and does not require enormous data in few-shot incremental tasks. Extensive experiments on three benchmark datasets validate that ASP consistently outperforms state-of-the-art FSCIL and prompt-based CIL methods in terms of both learning new classes and mitigating forgetting.

Key facts. Cited by 15 papers in the map (data through 7 Oct 2026). Builds on 49 of its 64 references that are in the map. First posted on arXiv 14 Mar 2024.

Builds on

49 of 64 references.

References · 49

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  3. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  4. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,160
  5. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,302
  6. 2015Distilling the Knowledge in a Neural Networkcited by 10,076
  7. 2016Overcoming catastrophic forgetting in neural networkscited by 4,718
  8. 2021The Power of Scale for Parameter-Efficient Prompt Tuningcited by 3,769
  9. 2017Prototypical Networks for Few-shot Learningcited by 3,001
  10. 2016Learning without Forgettingcited by 2,465
  11. 2016iCaRL: Incremental Classifier and Representation Learningcited by 2,313
  12. 2020The Many Faces of Robustness: A Critical Analysis of Out-of-Distribution Generalizationcited by 1,842
37 more
  1. 2022Visual Prompt Tuningcited by 1,674
  2. 2017Memory Aware Synapses: Learning what (not) to forgetcited by 1,094
  3. 2018Experience Replay for Continual Learningcited by 1,074
  4. 2016An overview of gradient descent optimization algorithmscited by 1,021
  5. 2021Learning to Prompt for Continual Learningcited by 938
  6. 2016Deep Variational Information Bottleneckcited by 915
  7. 2019Large Scale Incremental Learningcited by 727
  8. 2022DualPrompt: Complementary Prompting for Rehearsal-free Continual Learningcited by 635
  9. 2019Three scenarios for continual learningcited by 610
  10. 2019Gradient based sample selection for online continual learningcited by 568
  11. 2020Class-incremental learning: survey and performance evaluation on image classificationcited by 446
  12. 2022CODA-Prompt: COntinual Decomposed Attention-based Prompting for Rehearsal-Free Continual Learningcited by 446
  13. 2020Few-Shot Class-Incremental Learningcited by 249
  14. 2022FOSTER: Feature Boosting and Compression for Class-Incremental Learningcited by 228
  15. 2021Few-Shot Incremental Learning with Continually Evolved Classifierscited by 184
  16. 2023Revisiting Class-Incremental Learning with Pre-Trained Models: Generalizability and Adaptivity are All You Needcited by 183
  17. 2022Forward Compatible Few-Shot Class-Incremental Learningcited by 125
  18. 2023Hierarchical Decomposition of Prompt-Based Continual Learning: Rethinking Obscured Sub-optimalitycited by 121
  19. 2021Overcoming Catastrophic Forgetting in Incremental Few-Shot Learning by Finding Flat Minimacited by 118
  20. 2021Semantic-aware Knowledge Distillation for Few-Shot Class-Incremental Learningcited by 114
  21. 2021Self-Promoted Prototype Refinement for Few-Shot Class-Incremental Learningcited by 95
  22. 2023FeCAM: Exploiting the Heterogeneity of Class Distributions in Exemplar-Free Continual Learningcited by 86
  23. 2022Few-Shot Class-Incremental Learning from an Open-Set Perspectivecited by 83
  24. 2023A Comprehensive Survey of Forgetting in Deep Learning Beyond Continual Learningcited by 66
  25. 2023PILOT: A Pre-Trained Model-Based Continual Learning Toolboxcited by 64
  26. 2023Learning with Fantasy: Semantic-Aware Virtual Contrastive Constraint for Few-Shot Class-Incremental Learningcited by 55
  27. 2023A Survey on Few-Shot Class-Incremental Learningcited by 48
  28. 2019Rehearsal-Free Continual Learning over Small Non-I.I.D. Batchescited by 46
  29. 2023Few-Shot Class-Incremental Learning via Training-Free Prototype Calibrationcited by 44
  30. 2021Generalized and Incremental Few-Shot Learning by Explicit Learning and Calibration without Forgettingcited by 33
  31. 2022Grow and Merge: A Unified Framework for Continuous Categories Discoverycited by 33
  32. 2023Multimodal Parameter-Efficient Few-Shot Class Incremental Learningcited by 19
  33. 2022Meta-Learning with Less Forgetting on Large-Scale Non-Stationary Task Distributionscited by 14
  34. 2023DEJA VU: Continual Model Generalization For Unseen Domainscited by 13
  35. 2024Learning Prompt with Distribution-Based Feature Replay for Few-Shot Class-Incremental Learningcited by 12
  36. 2023Image-Object-Specific Prompt Learning for Few-Shot Class-Incremental Learningcited by 4
  37. 2023Federated Continual Novel Class Learningcited by 4

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 3; 2025: 7; 2026: 5.

15 papers.

Sort:

By area:

  1. 2024A Watermark for Order-Agnostic Language Modelscited by 14
  2. 2024De-mark: Watermark Removal in Large Language Modelscited by 11
  3. 2025Modality-Balancing Preference Optimization of Large Multimodal Models by Adversarial Negative Miningcited by 8
  4. 2025Revisiting Pool-based Prompt Learning for Few-shot Class-incremental Learningcited by 8
  5. 2024Towards Incremental Learning in Large Language Models: A Critical Reviewcited by 6
  6. 2025Latest Advancements Towards Catastrophic Forgetting under Data Scarcity: A Comprehensive Survey on Few-Shot Class Incremental Learningcited by 3
  7. 2025Continuous Knowledge-Preserving Decomposition with Adaptive Layer Selection for Few-Shot Class-Incremental Learningcited by 1
  8. 2025Adaptive Additive Parameter Updates of Vision Transformers for Few-Shot Continual Learningcited by 0
  9. 2025Sculpting Margin Penalty: Intra-Task Adapter Merging and Classifier Calibration for Few-Shot Class-Incremental Learningcited by 0
  10. 2025SAFA-SNN: Sparsity-Aware On-Device Few-Shot Class-Incremental Learning with Fast-Adaptive Structure of Spiking Neural Networkcited by 0
  11. 2026CASP: Few-Shot Class-Incremental Learning with CLS Token Attention Steering Promptscited by 0
  12. 2026Unlocking Prototype Potential: An Efficient Tuning Framework for Few-Shot Class-Incremental Learningcited by 0
3 more
  1. 2026Few-Shot Domain Incremental Learning via Continual Vision-Language Consolidationcited by 0
  2. 2026Decoupled and Distilled: Task-Adaptive LoRA-Teachers with Ensemble Knowledge Transfer for Few-Shot Class-Incremental Learningcited by 0
  3. 2026Dynamic LoRA-Experts and Prototype-Ensemble Matching for Class-Incremental Learningcited by 0