mlsanityData to 7 Oct 2026

Mamba-FSCIL: Dynamic Adaptation with Selective State Space Model for Few-Shot Class-Incremental Learning

arXiv 2407.06136 ↗First posted 8 Jul 2024cs.CVcited by 7

On the map: State-space sequence models (S4, Mamba) → Core SSM architectures for language

Abstract (arXiv)

Few-shot class-incremental learning (FSCIL) aims to incrementally learn novel classes from limited examples while preserving knowledge of previously learned classes. Existing methods face a critical dilemma: static architectures rely on a constant parameter space to learn from data that arrive sequentially, making them prone to overfitting to the current session, while dynamic architectures continually expand the parameter space, leading to increased complexity. In this study, we explore the potential of Selective State Space Models (SSMs) for FSCIL. Mamba leverages its input-dependent parameters to dynamically adjust its processing patterns and generate content-aware scan patterns without session-wise projector expansion. This enables it to configure distinct processing for base and novel classes, helping preserve existing knowledge while adapting to new ones. To leverage Mamba's potential for FSCIL, we design two key modules: First, we propose a dual selective SSM projector that generates input-conditioned state-space parameters from intermediate features for dynamic adaptation. The dual design structurally decouples base and novel-class processing, employing a frozen base branch to maintain stable base-class features and a dynamic incremental branch that adaptively learns distinctive feature shifts for novel classes. Second, we develop a class-sensitive selective scan mechanism to guide dynamic adaptation of the incremental branch. It reduces the disruption to base-class representations caused by training on novel data, and meanwhile, encourages the selective scan to perform in distinct patterns between base and novel classes. Extensive experiments on miniImageNet, CIFAR-100, and CUB-200 demonstrate that Mamba-FSCIL achieves state-of-the-art performance.

Key facts. Cited by 7 papers in the map (data through 7 Oct 2026). Placed in Core SSM architectures for language, one of 10 directions in State-space sequence models (S4, Mamba). Builds on 107 of its 148 references that are in the map. First posted on arXiv 8 Jul 2024.

Builds on

107 of 148 references.

Closest ancestors · 6

Same direction, newest first

  1. 2024An Empirical Study of Mamba-based Language Models12 Jun
  2. 2024Transformers are SSMs: Generalized Models and Efficient Algorithms Through Structured State Space Duality31 May
  3. 2023Mamba: Linear-Time Sequence Modeling with Selective State Spaces1 Dec
  4. 2023Structured State Space Models for In-Context Reinforcement Learning7 Mar
  5. 2022Simplified State Space Layers for Sequence Modeling9 Aug
  6. 2021Efficiently Modeling Long Sequences with Structured State Spaces31 Oct

Other references · 101

  1. 2017Attention Is All You Needcited by 59,932
  2. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  3. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  4. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  5. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,084
  6. 2021Swin Transformer: Hierarchical Vision Transformer using Shifted Windowscited by 12,082
  7. 2014ImageNet Large Scale Visual Recognition Challengecited by 8,441
  8. 2021Masked Autoencoders Are Scalable Vision Learnerscited by 7,339
  9. 2016Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localizationcited by 6,690
  10. 2017Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networkscited by 4,861
  11. 2016Overcoming catastrophic forgetting in neural networkscited by 4,718
  12. 2014Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translationcited by 4,016
38 more
  1. 2017Prototypical Networks for Few-shot Learningcited by 3,001
  2. 2016Learning without Forgettingcited by 2,465
  3. 2016iCaRL: Incremental Classifier and Representation Learningcited by 2,313
  4. 2016Matching Networks for One Shot Learningcited by 2,246
  5. 2017Gradient Episodic Memory for Continual Learningcited by 1,871
  6. 2017Learning to Compare: Relation Network for Few-Shot Learningcited by 1,139
  7. 2017Memory Aware Synapses: Learning what (not) to forgetcited by 1,094
  8. 2024Vision Mamba: Efficient Visual Representation Learning with Bidirectional State Space Modelcited by 1,051
  9. 2018Efficient Lifelong Learning with A-GEMcited by 1,044
  10. 2018On First-Order Meta-Learning Algorithmscited by 1,003
  11. 2024VMamba: Visual State Space Modelcited by 955
  12. 2023A Comprehensive Survey of Continual Learning: Theory, Method and Applicationcited by 860
  13. 2023RWKV: Reinventing RNNs for the Transformer Eracited by 814
  14. 2019Large Scale Incremental Learningcited by 727
  15. 2020Prevalence of Neural Collapse during the terminal phase of deep learning trainingcited by 713
  16. 2013An Empirical Investigation of Catastrophic Forgetting in Gradient-Based Neural Networkscited by 711
  17. 2017Lifelong Learning with Dynamically Expandable Networkscited by 689
  18. 2019A Closer Look at Few-shot Classificationcited by 679
  19. 2021Combining Recurrent, Convolutional, and Continuous-time Models with Linear State-Space Layerscited by 607
  20. 2020HiPPO: Recurrent Memory with Optimal Polynomial Projectionscited by 600
  21. 2023Retentive Network: A Successor to Transformer for Large Language Modelscited by 594
  22. 2022Hungry Hungry Hippos: Towards Language Modeling with State Space Modelscited by 495
  23. 2020Class-incremental learning: survey and performance evaluation on image classificationcited by 446
  24. 2022Masked Autoencoders As Spatiotemporal Learnerscited by 428
  25. 2019Dreaming to Distill: Data-free Knowledge Transfer via DeepInversioncited by 419
  26. 2024U-Mamba: Enhancing Long-range Dependency for Biomedical Image Segmentationcited by 412
  27. 2023Hyena Hierarchy: Towards Larger Convolutional Language Modelscited by 388
  28. 2021DER: Dynamically Expandable Representation for Class Incremental Learningcited by 355
  29. 2024VideoMamba: State Space Model for Efficient Video Understandingcited by 325
  30. 2021DyTox: Transformers for Continual Learning with DYnamic TOken eXpansioncited by 291
  31. 2024VM-UNet: Vision Mamba UNet for Medical Image Segmentationcited by 286
  32. 2019Maintaining Discrimination and Fairness in Class Incremental Learningcited by 281
  33. 2017Meta Networkscited by 268
  34. 2024SegMamba: Long-range Sequential Modeling Mamba For 3D Medical Image Segmentationcited by 266
  35. 2020Few-Shot Class-Incremental Learningcited by 249
  36. 2024Reinforcement learningcited by 236
  37. 2022FOSTER: Feature Boosting and Compression for Class-Incremental Learningcited by 228
  38. 2023Class-Incremental Learning: A Surveycited by 209

50 most cited of 101 All 101 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 2; 2025: 4; 2026: 1.

7 papers.

Similar, not cited

Close in wording, no citation either way

  1. 2026A Theoretical Analysis of Mamba's Training Dynamics: Filtering Relevant Features for Generalization in State Space Models
  2. 2024Mamba-CL: Optimizing Selective State Space Model in Null Space for Continual Learning
  3. 2026Learning to Curate What You Generate for Generalizable Few-Shot Class-Incremental Learning
  4. 2026Feature-Space Generative Models for One-Shot Class-Incremental Learning
  5. 2025Block-Biased Mamba for Long-Range Sequence Processing
  6. 2025Mamba-Adaptor: State Space Model Adaptor for Visual Recognition
  7. 2024Parameter Efficient Mamba Tuning via Projector-targeted Diagonal-centric Linear Transformation
  8. 2025Understanding Input Selectivity in Mamba: Impact on Approximation Power, Memorization, and Associative Recall Capacity