# Learning to Continually Learn

arXiv: https://arxiv.org/abs/2002.09571 · HTML: https://mlsanity.com/paper/2002.09571/

## Key facts

- Cited by 84 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 21 Feb 2020.

## Built on by (most cited first, 30 of 84)

- [A Comprehensive Survey of Continual Learning: Theory, Method and Application](https://mlsanity.com/paper/2302.00487/) (2023)
- [Constrained Few-shot Class-incremental Learning](https://mlsanity.com/paper/2203.16588/) (2022)
- [Advances and Challenges in Meta-Learning: A Technical Review](https://mlsanity.com/paper/2307.04722/) (2023)
- [Wide Neural Networks Forget Less Catastrophically](https://mlsanity.com/paper/2110.11526/) (2021)
- [Recent Advances of Continual Learning in Computer Vision: An Overview](https://mlsanity.com/paper/2109.11369/) (2021)
- [Meta-DMoE: Adapting to Domain Shift by Meta-Distillation from Mixture-of-Experts](https://mlsanity.com/paper/2210.03885/) (2022)
- [Recent Advances of Foundation Language Models-based Continual Learning: A Survey](https://mlsanity.com/paper/2405.18653/) (2024)
- [Improving Language Plasticity via Pretraining with Active Forgetting](https://mlsanity.com/paper/2307.01163/) (2023)
- [Continual Learning for Recurrent Neural Networks: an Empirical Evaluation](https://mlsanity.com/paper/2103.07492/) (2021)
- [The Effect of Task Ordering in Continual Learning](https://mlsanity.com/paper/2205.13323/) (2022)
- [Memory-Efficient Semi-Supervised Continual Learning: The World is its Own Replay Buffer](https://mlsanity.com/paper/2101.09536/) (2021)
- [Building a Subspace of Policies for Scalable Continual Learning](https://mlsanity.com/paper/2211.10445/) (2022)
- [How to Reuse and Compose Knowledge for a Lifetime of Tasks: A Survey on Continual Learning and Functional Composition](https://mlsanity.com/paper/2207.07730/) (2022)
- [Self-Paced Context Evaluation for Contextual Reinforcement Learning](https://mlsanity.com/paper/2106.05110/) (2021)
- [Optimizing Reusable Knowledge for Continual Learning via Metalearning](https://mlsanity.com/paper/2106.05390/) (2021)
- [Continual Object Detection: A review of definitions, strategies, and challenges](https://mlsanity.com/paper/2205.15445/) (2022)
- [Formalizing the Generalization-Forgetting Trade-off in Continual Learning](https://mlsanity.com/paper/2109.14035/) (2021)
- [Condensed Composite Memory Continual Learning](https://mlsanity.com/paper/2102.09890/) (2021)
- [Prototype-Guided Memory Replay for Continual Learning](https://mlsanity.com/paper/2108.12641/) (2021)
- [Convolutional Neural Networks Are Not Invariant to Translation, but They Can Learn to Be](https://mlsanity.com/paper/2110.05861/) (2021)
- [On-device Training: A First Overview on Existing Systems](https://mlsanity.com/paper/2212.00824/) (2022)
- [Continual Learning for Image-Based Camera Localization](https://mlsanity.com/paper/2108.09112/) (2021)
- [Is forgetting less a good inductive bias for forward transfer?](https://mlsanity.com/paper/2303.08207/) (2023)
- [Encoders and Ensembles for Task-Free Continual Learning](https://mlsanity.com/paper/2105.13327/) (2021)
- [ModuleFormer: Modularity Emerges from Mixture-of-Experts](https://mlsanity.com/paper/2306.04640/) (2023)
- [The Future of Continual Learning in the Era of Foundation Models: Three Key Directions](https://mlsanity.com/paper/2506.03320/) (2025)
- [Two Sparsities Are Better Than One: Unlocking the Performance Benefits of Sparse-Sparse Networks](https://mlsanity.com/paper/2112.13896/) (2021)
- [A contrastive rule for meta-learning](https://mlsanity.com/paper/2104.01677/) (2021)
- [Federated Reconnaissance: Efficient, Distributed, Class-Incremental Learning](https://mlsanity.com/paper/2109.00150/) (2021)
- [MoFO: Momentum-Filtered Optimizer for Mitigating Forgetting in LLM Fine-Tuning](https://mlsanity.com/paper/2407.20999/) (2024)

## Abstract (arXiv)

Continual lifelong learning requires an agent or model to learn many sequentially ordered tasks, building on previous knowledge without catastrophically forgetting it. Much work has gone towards preventing the default tendency of machine learning models to catastrophically forget, yet virtually all such work involves manually-designed solutions to the problem. We instead advocate meta-learning a solution to catastrophic forgetting, allowing AI to learn to continually learn. Inspired by neuromodulatory processes in the brain, we propose A Neuromodulated Meta-Learning Algorithm (ANML). It differentiates through a sequential learning process to meta-learn an activation-gating function that enables context-dependent selective activation within a deep neural network. Specifically, a neuromodulatory (NM) neural network gates the forward pass of another (otherwise normal) neural network called the prediction learning network (PLN). The NM network also thus indirectly controls selective plasticity (i.e. the backward pass of) the PLN. ANML enables continual learning without catastrophic forgetting at scale: it produces state-of-the-art continual learning performance, sequentially learning as many as 600 classes (over 9,000 SGD updates).