# Few-shot Continual Learning: a Brain-inspired Approach

arXiv: https://arxiv.org/abs/2104.09034 · HTML: https://mlsanity.com/paper/2104.09034/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 27 of its 35 references that are in the map.
- First posted on arXiv 19 Apr 2021.

## Builds on (most cited first, 27 of 27)

- [ImageNet Large Scale Visual Recognition Challenge](https://mlsanity.com/paper/1409.0575/) (2014)
- [Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks](https://mlsanity.com/paper/1703.03400/) (2017)
- [Overcoming catastrophic forgetting in neural networks](https://mlsanity.com/paper/1612.00796/) (2016)
- [Prototypical Networks for Few-shot Learning](https://mlsanity.com/paper/1703.05175/) (2017)
- [iCaRL: Incremental Classifier and Representation Learning](https://mlsanity.com/paper/1611.07725/) (2016)
- [Matching Networks for One Shot Learning](https://mlsanity.com/paper/1606.04080/) (2016)
- [Improved Training of Wasserstein GANs](https://mlsanity.com/paper/1704.00028/) (2017)
- [Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks](https://mlsanity.com/paper/1511.06434/) (2015)
- [Continual Lifelong Learning with Neural Networks: A Review](https://mlsanity.com/paper/1802.07569/) (2018)
- [Memory Aware Synapses: Learning what (not) to forget](https://mlsanity.com/paper/1711.09601/) (2017)
- [Riemannian Walk for Incremental Learning: Understanding Forgetting and Intransigence](https://mlsanity.com/paper/1801.10112/) (2018)
- [A Closer Look at Few-shot Classification](https://mlsanity.com/paper/1904.04232/) (2019)
- [End-to-End Incremental Learning](https://mlsanity.com/paper/1807.09536/) (2018)
- [Meta-Learning for Semi-Supervised Few-Shot Classification](https://mlsanity.com/paper/1803.00676/) (2018)
- [Measuring Catastrophic Forgetting in Neural Networks](https://mlsanity.com/paper/1708.02072/) (2017)
- [How to train your MAML](https://mlsanity.com/paper/1810.09502/) (2018)
- [CORe50: a New Dataset and Benchmark for Continuous Object Recognition](https://mlsanity.com/paper/1705.03550/) (2017)
- [Few-Shot Class-Incremental Learning](https://mlsanity.com/paper/2004.10956/) (2020)
- [Meta-Learning Representations for Continual Learning](https://mlsanity.com/paper/1905.12588/) (2019)
- [Edge-labeling Graph Neural Network for Few-shot Learning](https://mlsanity.com/paper/1905.01436/) (2019)
- [Learning to Continually Learn](https://mlsanity.com/paper/2002.09571/) (2020)
- [Incremental Few-Shot Learning with Attention Attractor Networks](https://mlsanity.com/paper/1810.07218/) (2018)
- [Adaptive Posterior Learning: few-shot learning with a surprise-based memory module](https://mlsanity.com/paper/1902.02527/) (2019)
- [Defining Benchmarks for Continual Few-Shot Learning](https://mlsanity.com/paper/2004.11967/) (2020)
- [Incremental Meta-Learning via Indirect Discriminant Alignment](https://mlsanity.com/paper/2002.04162/) (2020)
- [Cognitively-Inspired Model for Incremental Learning Using a Few Examples](https://mlsanity.com/paper/2002.12411/) (2020)
- [Computational models of long term plasticity and memory](https://mlsanity.com/paper/1706.04946/) (2017)

## Built on by (most cited first, 4 of 4)

- [Continual HyperTransformer: A Meta-Learner for Continual Few-Shot Learning](https://mlsanity.com/paper/2301.04584/) (2023)
- [Expanding continual few-shot learning benchmarks to include recognition of specific instances](https://mlsanity.com/paper/2209.07863/) (2022)
- [Brain-inspired analogical mixture prototypes for few-shot class-incremental learning](https://mlsanity.com/paper/2502.18923/) (2025)
- [Small Data Explainer -- The impact of small data methods in everyday life](https://mlsanity.com/paper/2507.11773/) (2025)

## Abstract (arXiv)

It is an important yet challenging setting to continually learn new tasks from a few examples. Although numerous efforts have been devoted to either continual learning or few-shot learning, little work has considered this new setting of few-shot continual learning (FSCL), which needs to minimize the catastrophic forgetting to the old tasks and gradually improve the ability of few-shot generalization. In this paper, we provide a first systematic study on FSCL and present an effective solution with deep neural networks. Our solution is based on the observation that continual learning of a task sequence inevitably interferes few-shot generalization, which makes it highly nontrivial to extend few-shot learning strategies to continual learning scenarios. We draw inspirations from the robust brain system and develop a method that (1) interdependently updates a pair of fast / slow weights for continual learning and few-shot learning to disentangle their divergent objectives, inspired by the biological model of meta-plasticity and fast / slow synapse; and (2) applies a brain-inspired two-step consolidation strategy to learn a task sequence without forgetting in the fast weights while improve generalization without overfitting in the slow weights. Extensive results on various benchmarks show that our method achieves a better performance than joint training of all the tasks ever seen. The ability of few-shot generalization is also substantially improved from incoming tasks and examples.