# Defining Benchmarks for Continual Few-Shot Learning

arXiv: https://arxiv.org/abs/2004.11967 · HTML: https://mlsanity.com/paper/2004.11967/

## Key facts

- Cited by 21 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 15 Apr 2020.

## Built on by (most cited first, 21 of 21)

- [Learn Continually, Generalize Rapidly: Lifelong Knowledge Accumulation for Few-shot Learning](https://mlsanity.com/paper/2104.08808/) (2021)
- [Continual Training of Language Models for Few-Shot Learning](https://mlsanity.com/paper/2210.05549/) (2022)
- [Continuous Coordination As a Realistic Scenario for Lifelong Learning](https://mlsanity.com/paper/2103.03216/) (2021)
- [Few-Shot Learning with Class Imbalance](https://mlsanity.com/paper/2101.02523/) (2021)
- [CL-MASR: A Continual Learning Benchmark for Multilingual ASR](https://mlsanity.com/paper/2310.16931/) (2023)
- [CoMPS: Continual Meta Policy Search](https://mlsanity.com/paper/2112.04467/) (2021)
- [Memory Efficient Meta-Learning with Large Images](https://mlsanity.com/paper/2107.01105/) (2021)
- [Meta-Learning with Less Forgetting on Large-Scale Non-Stationary Task Distributions](https://mlsanity.com/paper/2209.01501/) (2022)
- [Meta Learning on a Sequence of Imbalanced Domains with Difficulty Awareness](https://mlsanity.com/paper/2109.14120/) (2021)
- [Approximate Bayesian Class-Conditional Models under Continuous Representation Shift](https://mlsanity.com/paper/2305.19076/) (2023)
- [Few-shot Continual Learning: a Brain-inspired Approach](https://mlsanity.com/paper/2104.09034/) (2021)
- [Schedule-Robust Online Continual Learning](https://mlsanity.com/paper/2210.05561/) (2022)
- [Continual HyperTransformer: A Meta-Learner for Continual Few-Shot Learning](https://mlsanity.com/paper/2301.04584/) (2023)
- [Chunking: Continual Learning is not just about Distribution Shift](https://mlsanity.com/paper/2310.02206/) (2023)
- [Hyperparameter Selection in Continual Learning](https://mlsanity.com/paper/2404.06466/) (2024)
- [One-shot learning for the long term: consolidation with an artificial hippocampal algorithm](https://mlsanity.com/paper/2102.07503/) (2021)
- [How Sensitive are Meta-Learners to Dataset Imbalance?](https://mlsanity.com/paper/2104.05344/) (2021)
- [Variational Autoencoder with Disentanglement Priors for Low-Resource Task-Specific Natural Language Generation](https://mlsanity.com/paper/2202.13363/) (2022)
- [One-Shot Domain Incremental Learning](https://mlsanity.com/paper/2403.16707/) (2024)
- [Prospective Learning in Retrospect](https://mlsanity.com/paper/2507.07965/) (2025)
- [Active perception and disentangled representations allow continual, episodic zero and few-shot learning](https://mlsanity.com/paper/2602.19355/) (2026)

## Abstract (arXiv)

Both few-shot and continual learning have seen substantial progress in the last years due to the introduction of proper benchmarks. That being said, the field has still to frame a suite of benchmarks for the highly desirable setting of continual few-shot learning, where the learner is presented a number of few-shot tasks, one after the other, and then asked to perform well on a validation set stemming from all previously seen tasks. Continual few-shot learning has a small computational footprint and is thus an excellent setting for efficient investigation and experimentation. In this paper we first define a theoretical framework for continual few-shot learning, taking into account recent literature, then we propose a range of flexible benchmarks that unify the evaluation criteria and allows exploring the problem from multiple perspectives. As part of the benchmark, we introduce a compact variant of ImageNet, called SlimageNet64, which retains all original 1000 classes but only contains 200 instances of each one (a total of 200K data-points) downscaled to 64 x 64 pixels. We provide baselines for the proposed benchmarks using a number of popular few-shot learning algorithms, as a result, exposing previously unknown strengths and weaknesses of those algorithms in continual and data-limited settings.