# Few-Shot Class-Incremental Learning by Sampling Multi-Phase Tasks

arXiv: https://arxiv.org/abs/2203.17030 · HTML: https://mlsanity.com/paper/2203.17030/

## Key facts

- Cited by 88 papers in the map (data through 14 Oct 2026).
- Builds on 67 of its 111 references that are in the map.
- First posted on arXiv 31 Mar 2022.

## Builds on (most cited first, 30 of 67)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [Distilling the Knowledge in a Neural Network](https://mlsanity.com/paper/1503.02531/) (2015)
- [ImageNet Large Scale Visual Recognition Challenge](https://mlsanity.com/paper/1409.0575/) (2014)
- [End-to-End Object Detection with Transformers](https://mlsanity.com/paper/2005.12872/) (2020)
- [Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks](https://mlsanity.com/paper/1703.03400/) (2017)
- [Overcoming catastrophic forgetting in neural networks](https://mlsanity.com/paper/1612.00796/) (2016)
- [Prototypical Networks for Few-shot Learning](https://mlsanity.com/paper/1703.05175/) (2017)
- [Learning without Forgetting](https://mlsanity.com/paper/1606.09282/) (2016)
- [iCaRL: Incremental Classifier and Representation Learning](https://mlsanity.com/paper/1611.07725/) (2016)
- [Matching Networks for One Shot Learning](https://mlsanity.com/paper/1606.04080/) (2016)
- [Pyramid Vision Transformer: A Versatile Backbone for Dense Prediction without Convolutions](https://mlsanity.com/paper/2102.12122/) (2021)
- [Gradient Episodic Memory for Continual Learning](https://mlsanity.com/paper/1706.08840/) (2017)
- [Deep Sets](https://mlsanity.com/paper/1703.06114/) (2017)
- [Learning to Compare: Relation Network for Few-Shot Learning](https://mlsanity.com/paper/1711.06025/) (2017)
- [Memory Aware Synapses: Learning what (not) to forget](https://mlsanity.com/paper/1711.09601/) (2017)
- [Efficient Lifelong Learning with A-GEM](https://mlsanity.com/paper/1812.00420/) (2018)
- [On First-Order Meta-Learning Algorithms](https://mlsanity.com/paper/1803.02999/) (2018)
- [Transformers in Vision: A Survey](https://mlsanity.com/paper/2101.01169/) (2021)
- [Generalized Out-of-Distribution Detection: A Survey](https://mlsanity.com/paper/2110.11334/) (2021)
- [Large Scale Incremental Learning](https://mlsanity.com/paper/1905.13260/) (2019)
- [An Empirical Investigation of Catastrophic Forgetting in Gradient-Based Neural Networks](https://mlsanity.com/paper/1312.6211/) (2013)
- [Lifelong Learning with Dynamically Expandable Networks](https://mlsanity.com/paper/1708.01547/) (2017)
- [A Closer Look at Few-shot Classification](https://mlsanity.com/paper/1904.04232/) (2019)
- [End-to-End Incremental Learning](https://mlsanity.com/paper/1807.09536/) (2018)
- [Generalizing from a Few Examples: A Survey on Few-Shot Learning](https://mlsanity.com/paper/1904.05046/) (2019)

## Built on by (most cited first, 30 of 88)

- [A Comprehensive Survey of Continual Learning: Theory, Method and Application](https://mlsanity.com/paper/2302.00487/) (2023)
- [FOSTER: Feature Boosting and Compression for Class-Incremental Learning](https://mlsanity.com/paper/2204.04662/) (2022)
- [Class-Incremental Learning: A Survey](https://mlsanity.com/paper/2302.03648/) (2023)
- [Revisiting Class-Incremental Learning with Pre-Trained Models: Generalizability and Adaptivity are All You Need](https://mlsanity.com/paper/2303.07338/) (2023)
- [A Model or 603 Exemplars: Towards Memory-Efficient Class-Incremental Learning](https://mlsanity.com/paper/2205.13218/) (2022)
- [Neural Collapse Inspired Feature-Classifier Alignment for Few-Shot Class Incremental Learning](https://mlsanity.com/paper/2302.03004/) (2023)
- [FeCAM: Exploiting the Heterogeneity of Class Distributions in Exemplar-Free Continual Learning](https://mlsanity.com/paper/2309.14062/) (2023)
- [A Comprehensive Survey of Forgetting in Deep Learning Beyond Continual Learning](https://mlsanity.com/paper/2307.09218/) (2023)
- [Learning with Fantasy: Semantic-Aware Virtual Contrastive Constraint for Few-Shot Class-Incremental Learning](https://mlsanity.com/paper/2304.00426/) (2023)
- [No One Left Behind: Real-World Federated Class-Incremental Learning](https://mlsanity.com/paper/2302.00903/) (2023)
- [A Survey on Few-Shot Class-Incremental Learning](https://mlsanity.com/paper/2304.08130/) (2023)
- [Few-Shot Class-Incremental Learning via Training-Free Prototype Calibration](https://mlsanity.com/paper/2312.05229/) (2023)
- [On the Soft-Subnetwork for Few-shot Class Incremental Learning](https://mlsanity.com/paper/2209.07529/) (2022)
- [Pre-trained Vision and Language Transformers Are Few-Shot Incremental Learners](https://mlsanity.com/paper/2404.02117/) (2024)
- [OrCo: Towards Better Generalization via Orthogonality and Contrast for Few-Shot Class-Incremental Learning](https://mlsanity.com/paper/2403.18550/) (2024)
- [Few-shot Class-incremental Learning for Classification and Object Detection: A Survey](https://mlsanity.com/paper/2308.06764/) (2023)
- [Multimodal Parameter-Efficient Few-Shot Class Incremental Learning](https://mlsanity.com/paper/2303.04751/) (2023)
- [Neural Collapse Terminus: A Unified Solution for Class Incremental Learning and Its Variants](https://mlsanity.com/paper/2308.01746/) (2023)
- [Rethinking Few-shot Class-incremental Learning: Learning from Yourself](https://mlsanity.com/paper/2407.07468/) (2024)
- [Rethinking Few-Shot Class-Incremental Learning with Open-Set Hypothesis in Hyperbolic Geometry](https://mlsanity.com/paper/2207.09963/) (2022)
- [Multi-Label Continual Learning using Augmented Graph Convolutional Network](https://mlsanity.com/paper/2211.14763/) (2022)
- [When Meta-Learning Meets Online and Continual Learning: A Survey](https://mlsanity.com/paper/2311.05241/) (2023)
- [Learning Prompt with Distribution-Based Feature Replay for Few-Shot Class-Incremental Learning](https://mlsanity.com/paper/2401.01598/) (2024)
- [CLOSER: Towards Better Representation Learning for Few-Shot Class-Incremental Learning](https://mlsanity.com/paper/2410.05627/) (2024)
- [Continual Adapter Tuning with Semantic Shift Compensation for Class-Incremental Learning](https://mlsanity.com/paper/2403.19979/) (2024)
- [Calibrating Higher-Order Statistics for Few-Shot Class-Incremental Learning with Pre-trained Vision Transformers](https://mlsanity.com/paper/2404.06622/) (2024)
- [Revisiting Pool-based Prompt Learning for Few-shot Class-incremental Learning](https://mlsanity.com/paper/2507.09183/) (2025)
- [Contrastive Augmented Graph2Graph Memory Interaction for Few Shot Continual Learning](https://mlsanity.com/paper/2403.04140/) (2024)
- [NTK-Guided Few-Shot Class Incremental Learning](https://mlsanity.com/paper/2403.12486/) (2024)
- [Mamba-FSCIL: Dynamic Adaptation with Selective State Space Model for Few-Shot Class-Incremental Learning](https://mlsanity.com/paper/2407.06136/) (2024)

## Abstract (arXiv)

New classes arise frequently in our ever-changing world, e.g., emerging topics in social media and new types of products in e-commerce. A model should recognize new classes and meanwhile maintain discriminability over old classes. Under severe circumstances, only limited novel instances are available to incrementally update the model. The task of recognizing few-shot new classes without forgetting old classes is called few-shot class-incremental learning (FSCIL). In this work, we propose a new paradigm for FSCIL based on meta-learning by LearnIng Multi-phase Incremental Tasks (LIMIT), which synthesizes fake FSCIL tasks from the base dataset. The data format of fake tasks is consistent with the `real' incremental tasks, and we can build a generalizable feature space for the unseen tasks through meta-learning. Besides, LIMIT also constructs a calibration module based on transformer, which calibrates the old class classifiers and new class prototypes into the same scale and fills in the semantic gap. The calibration module also adaptively contextualizes the instance-specific embedding with a set-to-set function. LIMIT efficiently adapts to new classes and meanwhile resists forgetting over old classes. Experiments on three benchmark datasets (CIFAR100, miniImageNet, and CUB200) and large-scale dataset, i.e., ImageNet ILSVRC2012 validate that LIMIT achieves state-of-the-art performance.