# Constrained Few-shot Class-incremental Learning

arXiv: https://arxiv.org/abs/2203.16588 · HTML: https://mlsanity.com/paper/2203.16588/

## Key facts

- Cited by 94 papers in the map (data through 14 Oct 2026).
- Builds on 40 of its 57 references that are in the map.
- First posted on arXiv 30 Mar 2022.

## Builds on (most cited first, 30 of 40)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [ImageNet Large Scale Visual Recognition Challenge](https://mlsanity.com/paper/1409.0575/) (2014)
- [Overcoming catastrophic forgetting in neural networks](https://mlsanity.com/paper/1612.00796/) (2016)
- [Going Deeper with Convolutions](https://mlsanity.com/paper/1409.4842/) (2014)
- [Prototypical Networks for Few-shot Learning](https://mlsanity.com/paper/1703.05175/) (2017)
- [iCaRL: Incremental Classifier and Representation Learning](https://mlsanity.com/paper/1611.07725/) (2016)
- [Matching Networks for One Shot Learning](https://mlsanity.com/paper/1606.04080/) (2016)
- [Gradient Episodic Memory for Continual Learning](https://mlsanity.com/paper/1706.08840/) (2017)
- [Unsupervised Feature Learning via Non-Parametric Instance-level Discrimination](https://mlsanity.com/paper/1805.01978/) (2018)
- [Big Transfer (BiT): General Visual Representation Learning](https://mlsanity.com/paper/1912.11370/) (2019)
- [Large Scale Incremental Learning](https://mlsanity.com/paper/1905.13260/) (2019)
- [An Empirical Investigation of Catastrophic Forgetting in Gradient-Based Neural Networks](https://mlsanity.com/paper/1312.6211/) (2013)
- [Neural Turing Machines](https://mlsanity.com/paper/1410.5401/) (2014)
- [End-to-End Incremental Learning](https://mlsanity.com/paper/1807.09536/) (2018)
- [Rethinking Few-Shot Image Classification: a Good Embedding Is All You Need?](https://mlsanity.com/paper/2003.11539/) (2020)
- [Dynamic Few-Shot Visual Learning without Forgetting](https://mlsanity.com/paper/1804.09458/) (2018)
- [Memory Networks](https://mlsanity.com/paper/1410.3916/) (2014)
- [Maintaining Discrimination and Fairness in Class Incremental Learning](https://mlsanity.com/paper/1911.07053/) (2019)
- [Rainbow Memory: Continual Learning with a Memory of Diverse Samples](https://mlsanity.com/paper/2103.17230/) (2021)
- [Few-Shot Class-Incremental Learning](https://mlsanity.com/paper/2004.10956/) (2020)
- [Mnemonics Training: Multi-Class Incremental Learning without Forgetting](https://mlsanity.com/paper/2002.10211/) (2020)
- [Coresets via Bilevel Optimization for Continual Learning and Streaming](https://mlsanity.com/paper/2006.03875/) (2020)
- [Few-Shot Incremental Learning with Continually Evolved Classifiers](https://mlsanity.com/paper/2104.03047/) (2021)
- [Overcoming Catastrophic Forgetting in Incremental Few-Shot Learning by Finding Flat Minima](https://mlsanity.com/paper/2111.01549/) (2021)
- [Semantic-aware Knowledge Distillation for Few-Shot Class-Incremental Learning](https://mlsanity.com/paper/2103.04059/) (2021)
- [Latent Replay for Real-Time Continual Learning](https://mlsanity.com/paper/1912.01100/) (2019)
- [Learning to Continually Learn](https://mlsanity.com/paper/2002.09571/) (2020)
- [GAN Memory with No Forgetting](https://mlsanity.com/paper/2006.07543/) (2020)
- [Incremental Few-Shot Learning with Attention Attractor Networks](https://mlsanity.com/paper/1810.07218/) (2018)
- [A Neuro-vector-symbolic Architecture for Solving Raven's Progressive Matrices](https://mlsanity.com/paper/2203.04571/) (2022)

## Built on by (most cited first, 30 of 94)

- [A Comprehensive Survey of Continual Learning: Theory, Method and Application](https://mlsanity.com/paper/2302.00487/) (2023)
- [Neural Collapse Inspired Feature-Classifier Alignment for Few-Shot Class Incremental Learning](https://mlsanity.com/paper/2302.03004/) (2023)
- [A Comprehensive Survey of Forgetting in Deep Learning Beyond Continual Learning](https://mlsanity.com/paper/2307.09218/) (2023)
- [Learning with Fantasy: Semantic-Aware Virtual Contrastive Constraint for Few-Shot Class-Incremental Learning](https://mlsanity.com/paper/2304.00426/) (2023)
- [Training of Physical Neural Networks](https://mlsanity.com/paper/2406.03372/) (2024)
- [A Survey on Few-Shot Class-Incremental Learning](https://mlsanity.com/paper/2304.08130/) (2023)
- [First Session Adaptation: A Strong Replay-Free Baseline for Class-Incremental Learning](https://mlsanity.com/paper/2303.13199/) (2023)
- [On the Soft-Subnetwork for Few-shot Class Incremental Learning](https://mlsanity.com/paper/2209.07529/) (2022)
- [Pre-trained Vision and Language Transformers Are Few-Shot Incremental Learners](https://mlsanity.com/paper/2404.02117/) (2024)
- [OrCo: Towards Better Generalization via Orthogonality and Contrast for Few-Shot Class-Incremental Learning](https://mlsanity.com/paper/2403.18550/) (2024)
- [Few-shot Class-incremental Learning for Classification and Object Detection: A Survey](https://mlsanity.com/paper/2308.06764/) (2023)
- [Torchhd: An Open Source Python Library to Support Research on Hyperdimensional Computing and Vector Symbolic Architectures](https://mlsanity.com/paper/2205.09208/) (2022)
- [Image-free Classifier Injection for Zero-Shot Classification](https://mlsanity.com/paper/2308.10599/) (2023)
- [Dynamics-Adaptive Continual Reinforcement Learning via Progressive Contextualization](https://mlsanity.com/paper/2209.00347/) (2022)
- [Deep Learning for Cross-Domain Few-Shot Visual Recognition: A Survey](https://mlsanity.com/paper/2303.08557/) (2023)
- [Neural Collapse Terminus: A Unified Solution for Class Incremental Learning and Its Variants](https://mlsanity.com/paper/2308.01746/) (2023)
- [Rethinking Few-shot Class-incremental Learning: Learning from Yourself](https://mlsanity.com/paper/2407.07468/) (2024)
- [Non-Exemplar Online Class-incremental Continual Learning via Dual-prototype Self-augment and Refinement](https://mlsanity.com/paper/2303.10891/) (2023)
- [Rethinking Few-Shot Class-Incremental Learning with Open-Set Hypothesis in Hyperbolic Geometry](https://mlsanity.com/paper/2207.09963/) (2022)
- [When Meta-Learning Meets Online and Continual Learning: A Survey](https://mlsanity.com/paper/2311.05241/) (2023)
- [Learning Prompt with Distribution-Based Feature Replay for Few-Shot Class-Incremental Learning](https://mlsanity.com/paper/2401.01598/) (2024)
- [CLOSER: Towards Better Representation Learning for Few-Shot Class-Incremental Learning](https://mlsanity.com/paper/2410.05627/) (2024)
- [In-memory Realization of In-situ Few-shot Continual Learning with a Dynamically Evolving Explicit Memory](https://mlsanity.com/paper/2207.06810/) (2022)
- [Calibrating Higher-Order Statistics for Few-Shot Class-Incremental Learning with Pre-trained Vision Transformers](https://mlsanity.com/paper/2404.06622/) (2024)
- [Revisiting Pool-based Prompt Learning for Few-shot Class-incremental Learning](https://mlsanity.com/paper/2507.09183/) (2025)
- [Expert-Guided Explainable Few-Shot Learning for Medical Image Diagnosis](https://mlsanity.com/paper/2509.08007/) (2025)
- [Learning from Hypervectors: A Survey on Hypervector Encoding](https://mlsanity.com/paper/2308.00685/) (2023)
- [HyperSense: Hyperdimensional Intelligent Sensing for Energy-Efficient Sparse Data Processing](https://mlsanity.com/paper/2401.10267/) (2024)
- [Contrastive Augmented Graph2Graph Memory Interaction for Few Shot Continual Learning](https://mlsanity.com/paper/2403.04140/) (2024)
- [NTK-Guided Few-Shot Class Incremental Learning](https://mlsanity.com/paper/2403.12486/) (2024)

## Abstract (arXiv)

Continually learning new classes from fresh data without forgetting previous knowledge of old classes is a very challenging research problem. Moreover, it is imperative that such learning must respect certain memory and computational constraints such as (i) training samples are limited to only a few per class, (ii) the computational cost of learning a novel class remains constant, and (iii) the memory footprint of the model grows at most linearly with the number of classes observed. To meet the above constraints, we propose C-FSCIL, which is architecturally composed of a frozen meta-learned feature extractor, a trainable fixed-size fully connected layer, and a rewritable dynamically growing memory that stores as many vectors as the number of encountered classes. C-FSCIL provides three update modes that offer a trade-off between accuracy and compute-memory cost of learning novel classes. C-FSCIL exploits hyperdimensional embedding that allows to continually express many more classes than the fixed dimensions in the vector space, with minimal interference. The quality of class vector representations is further improved by aligning them quasi-orthogonally to each other by means of novel loss functions. Experiments on the CIFAR100, miniImageNet, and Omniglot datasets show that C-FSCIL outperforms the baselines with remarkable accuracy and compression. It also scales up to the largest problem size ever tried in this few-shot setting by learning 423 novel classes on top of 1200 base classes with less than 1.6% accuracy drop. Our code is available at https://github.com/IBM/constrained-FSCIL.