# A Bag of Tricks for Few-Shot Class-Incremental Learning

arXiv: https://arxiv.org/abs/2403.14392 · HTML: https://mlsanity.com/paper/2403.14392/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 30 of its 41 references that are in the map.
- First posted on arXiv 21 Mar 2024.

## Builds on (most cited first, 30 of 30)

- [A Simple Framework for Contrastive Learning of Visual Representations](https://mlsanity.com/paper/2002.05709/) (2020)
- [ImageNet Large Scale Visual Recognition Challenge](https://mlsanity.com/paper/1409.0575/) (2014)
- [Supervised Contrastive Learning](https://mlsanity.com/paper/2004.11362/) (2020)
- [Learning without Forgetting](https://mlsanity.com/paper/1606.09282/) (2016)
- [iCaRL: Incremental Classifier and Representation Learning](https://mlsanity.com/paper/1611.07725/) (2016)
- [The Lottery Ticket Hypothesis: Finding Sparse, Trainable Neural Networks](https://mlsanity.com/paper/1803.03635/) (2018)
- [Unsupervised Representation Learning by Predicting Image Rotations](https://mlsanity.com/paper/1803.07728/) (2018)
- [Experience Replay for Continual Learning](https://mlsanity.com/paper/1811.11682/) (2018)
- [Prevalence of Neural Collapse during the terminal phase of deep learning training](https://mlsanity.com/paper/2008.08186/) (2020)
- [End-to-End Incremental Learning](https://mlsanity.com/paper/1807.09536/) (2018)
- [Class-incremental learning: survey and performance evaluation on image classification](https://mlsanity.com/paper/2010.15277/) (2020)
- [Few-Shot Class-Incremental Learning](https://mlsanity.com/paper/2004.10956/) (2020)
- [Few-Shot Incremental Learning with Continually Evolved Classifiers](https://mlsanity.com/paper/2104.03047/) (2021)
- [Rotate your Networks: Better Weight Consolidation and Less Catastrophic Forgetting](https://mlsanity.com/paper/1802.02950/) (2018)
- [Forward Compatible Few-Shot Class-Incremental Learning](https://mlsanity.com/paper/2203.06953/) (2022)
- [Overcoming Catastrophic Forgetting in Incremental Few-Shot Learning by Finding Flat Minima](https://mlsanity.com/paper/2111.01549/) (2021)
- [Neural Collapse Inspired Feature-Classifier Alignment for Few-Shot Class Incremental Learning](https://mlsanity.com/paper/2302.03004/) (2023)
- [Semantic-aware Knowledge Distillation for Few-Shot Class-Incremental Learning](https://mlsanity.com/paper/2103.04059/) (2021)
- [Self-Promoted Prototype Refinement for Few-Shot Class-Incremental Learning](https://mlsanity.com/paper/2107.08918/) (2021)
- [Constrained Few-shot Class-incremental Learning](https://mlsanity.com/paper/2203.16588/) (2022)
- [Few-Shot Class-Incremental Learning by Sampling Multi-Phase Tasks](https://mlsanity.com/paper/2203.17030/) (2022)
- [Few-Shot Class-Incremental Learning from an Open-Set Perspective](https://mlsanity.com/paper/2208.00147/) (2022)
- [Few-Shot Class-Incremental Learning via Entropy-Regularized Data-Free Replay](https://mlsanity.com/paper/2207.11213/) (2022)
- [MgSvF: Multi-Grained Slow vs. Fast Framework for Few-Shot Class-Incremental Learning](https://mlsanity.com/paper/2006.15524/) (2020)
- [Few-Shot Lifelong Learning](https://mlsanity.com/paper/2103.00991/) (2021)
- [Learning with Fantasy: Semantic-Aware Virtual Contrastive Constraint for Few-Shot Class-Incremental Learning](https://mlsanity.com/paper/2304.00426/) (2023)
- [Mimicking the Oracle: An Initial Phase Decorrelation Approach for Class Incremental Learning](https://mlsanity.com/paper/2112.04731/) (2021)
- [Margin-Based Few-Shot Class-Incremental Learning with Class-Level Overfitting Mitigation](https://mlsanity.com/paper/2210.04524/) (2022)
- [On the Soft-Subnetwork for Few-shot Class Incremental Learning](https://mlsanity.com/paper/2209.07529/) (2022)
- [Scaling Up Semi-supervised Learning with Unconstrained Unlabelled Data](https://mlsanity.com/paper/2306.01222/) (2023)

## Built on by (most cited first, 2 of 2)

- [Consistency-Guided Asynchronous Contrastive Tuning for Few-Shot Class-Incremental Tuning of Foundation Models](https://mlsanity.com/paper/2405.16625/) (2024)
- [Does Prior Data Matter? Exploring Joint Training in the Context of Few-Shot Class-Incremental Learning](https://mlsanity.com/paper/2503.10003/) (2025)

## Abstract (arXiv)

We present a bag of tricks framework for few-shot class-incremental learning (FSCIL), which is a challenging form of continual learning that involves continuous adaptation to new tasks with limited samples. FSCIL requires both stability and adaptability, i.e., preserving proficiency in previously learned tasks while learning new ones. Our proposed bag of tricks brings together six key and highly influential techniques that improve stability, adaptability, and overall performance under a unified framework for FSCIL. We organize these tricks into three categories: stability tricks, adaptability tricks, and training tricks. Stability tricks aim to mitigate the forgetting of previously learned classes by enhancing the separation between the embeddings of learned classes and minimizing interference when learning new ones. On the other hand, adaptability tricks focus on the effective learning of new classes. Finally, training tricks improve the overall performance without compromising stability or adaptability. We perform extensive experiments on three benchmark datasets, CIFAR-100, CUB-200, and miniIMageNet, to evaluate the impact of our proposed framework. Our detailed analysis shows that our approach substantially improves both stability and adaptability, establishing a new state-of-the-art by outperforming prior works in the area. We believe our method provides a go-to solution and establishes a robust baseline for future research in this area.