# Incremental Few-Shot Learning with Attention Attractor Networks

arXiv: https://arxiv.org/abs/1810.07218 · HTML: https://mlsanity.com/paper/1810.07218/

## Key facts

- Cited by 66 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 16 Oct 2018.

## Built on by (most cited first, 30 of 66)

- [Few-Shot Incremental Learning with Continually Evolved Classifiers](https://mlsanity.com/paper/2104.03047/) (2021)
- [Overcoming Catastrophic Forgetting in Incremental Few-Shot Learning by Finding Flat Minima](https://mlsanity.com/paper/2111.01549/) (2021)
- [Neural Collapse Inspired Feature-Classifier Alignment for Few-Shot Class Incremental Learning](https://mlsanity.com/paper/2302.03004/) (2023)
- [Semantic-aware Knowledge Distillation for Few-Shot Class-Incremental Learning](https://mlsanity.com/paper/2103.04059/) (2021)
- [Self-Promoted Prototype Refinement for Few-Shot Class-Incremental Learning](https://mlsanity.com/paper/2107.08918/) (2021)
- [Constrained Few-shot Class-incremental Learning](https://mlsanity.com/paper/2203.16588/) (2022)
- [Few-Shot Class-Incremental Learning by Sampling Multi-Phase Tasks](https://mlsanity.com/paper/2203.17030/) (2022)
- [Learning from Few Examples: A Summary of Approaches to Few-Shot Learning](https://mlsanity.com/paper/2203.04291/) (2022)
- [A Comprehensive Survey of Forgetting in Deep Learning Beyond Continual Learning](https://mlsanity.com/paper/2307.09218/) (2023)
- [RMM: Reinforced Memory Management for Class-Incremental Learning](https://mlsanity.com/paper/2301.05792/) (2023)
- [Few-Shot Lifelong Learning](https://mlsanity.com/paper/2103.00991/) (2021)
- [Subspace Regularizers for Few-Shot Class Incremental Learning](https://mlsanity.com/paper/2110.07059/) (2021)
- [Recent Advances of Continual Learning in Computer Vision: An Overview](https://mlsanity.com/paper/2109.11369/) (2021)
- [Attention, please! A survey of Neural Attention Models in Deep Learning](https://mlsanity.com/paper/2103.16775/) (2021)
- [Shallow Bayesian Meta Learning for Real-World Few-Shot Recognition](https://mlsanity.com/paper/2101.02833/) (2021)
- [A Survey on Few-Shot Class-Incremental Learning](https://mlsanity.com/paper/2304.08130/) (2023)
- [Graph Few-shot Class-incremental Learning](https://mlsanity.com/paper/2112.12819/) (2021)
- [On the Soft-Subnetwork for Few-shot Class Incremental Learning](https://mlsanity.com/paper/2209.07529/) (2022)
- [Generalized and Incremental Few-Shot Learning by Explicit Learning and Calibration without Forgetting](https://mlsanity.com/paper/2108.08165/) (2021)
- [GistNet: a Geometric Structure Transfer Network for Long-Tailed Recognition](https://mlsanity.com/paper/2105.00131/) (2021)
- [Incremental Few-shot Text Classification with Multi-round New Classes: Formulation, Dataset and System](https://mlsanity.com/paper/2104.11882/) (2021)
- [GP-Tree: A Gaussian Process Classifier for Few-Shot Incremental Learning](https://mlsanity.com/paper/2102.07868/) (2021)
- [Meta-learning approaches for few-shot learning: A survey of recent advances](https://mlsanity.com/paper/2303.07502/) (2023)
- [Open Long-Tailed Recognition in a Dynamic World](https://mlsanity.com/paper/2208.08349/) (2022)
- [Few-Shot Learning with Class Imbalance](https://mlsanity.com/paper/2101.02523/) (2021)
- [Reinforced Attention for Few-Shot Learning and Beyond](https://mlsanity.com/paper/2104.04192/) (2021)
- [Few-Shot Learning with a Strong Teacher](https://mlsanity.com/paper/2107.00197/) (2021)
- [Incremental Few-Shot Semantic Segmentation via Embedding Adaptive-Update and Hyper-class Representation](https://mlsanity.com/paper/2207.12964/) (2022)
- [Meta-Causal Feature Learning for Out-of-Distribution Generalization](https://mlsanity.com/paper/2208.10156/) (2022)
- [Neural Collapse Terminus: A Unified Solution for Class Incremental Learning and Its Variants](https://mlsanity.com/paper/2308.01746/) (2023)

## Abstract (arXiv)

Machine learning classifiers are often trained to recognize a set of pre-defined classes. However, in many applications, it is often desirable to have the flexibility of learning additional concepts, with limited data and without re-training on the full training set. This paper addresses this problem, incremental few-shot learning, where a regular classification network has already been trained to recognize a set of base classes, and several extra novel classes are being considered, each with only a few labeled examples. After learning the novel classes, the model is then evaluated on the overall classification performance on both base and novel classes. To this end, we propose a meta-learning model, the Attention Attractor Network, which regularizes the learning of novel classes. In each episode, we train a set of new weights to recognize novel classes until they converge, and we show that the technique of recurrent back-propagation can back-propagate through the optimization process and facilitate the learning of these parameters. We demonstrate that the learned attractor network can help recognize novel classes while remembering old classes without the need to review the original training set, outperforming various baselines.