# La-MAML: Look-ahead Meta Learning for Continual Learning

arXiv: https://arxiv.org/abs/2007.13904 · HTML: https://mlsanity.com/paper/2007.13904/

## Key facts

- Cited by 52 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 27 Jul 2020.

## Built on by (most cited first, 30 of 52)

- [Online Continual Learning in Image Classification: An Empirical Survey](https://mlsanity.com/paper/2101.10423/) (2021)
- [Remember the Past: Distilling Datasets into Addressable Memories for Neural Networks](https://mlsanity.com/paper/2206.02916/) (2022)
- [Continual Learning of Natural Language Processing Tasks: A Survey](https://mlsanity.com/paper/2211.12701/) (2022)
- [Flattening Sharpness for Dynamic Gradient Projection Memory Benefits Continual Learning](https://mlsanity.com/paper/2110.04593/) (2021)
- [A Comprehensive Survey of Forgetting in Deep Learning Beyond Continual Learning](https://mlsanity.com/paper/2307.09218/) (2023)
- [Reproducibility Report: La-MAML: Look-ahead Meta Learning for Continual Learning](https://mlsanity.com/paper/2102.05824/) (2021)
- [Continual Learning with Scaled Gradient Projection](https://mlsanity.com/paper/2302.01386/) (2023)
- [Learning where to learn: Gradient sparsity in meta and continual learning](https://mlsanity.com/paper/2110.14402/) (2021)
- [Continual Learning on Noisy Data Streams via Self-Purified Replay](https://mlsanity.com/paper/2110.07735/) (2021)
- [How to Reuse and Compose Knowledge for a Lifetime of Tasks: A Survey on Continual Learning and Functional Composition](https://mlsanity.com/paper/2207.07730/) (2022)
- [Formalizing the Generalization-Forgetting Trade-off in Continual Learning](https://mlsanity.com/paper/2109.14035/) (2021)
- [Sequoia: A Software Framework to Unify Continual Learning Research](https://mlsanity.com/paper/2108.01005/) (2021)
- [CoMPS: Continual Meta Policy Search](https://mlsanity.com/paper/2112.04467/) (2021)
- [Lifelong Learning Metrics](https://mlsanity.com/paper/2201.08278/) (2022)
- [Complementary Calibration: Boosting General Continual Learning with Collaborative Distillation and Self-Supervision](https://mlsanity.com/paper/2109.02426/) (2021)
- [Continuous Detection, Rapidly React: Unseen Rumors Detection based on Continual Prompt-Tuning](https://mlsanity.com/paper/2203.11720/) (2022)
- [Optimizing Class Distribution in Memory for Multi-Label Online Continual Learning](https://mlsanity.com/paper/2209.11469/) (2022)
- [One Pass ImageNet](https://mlsanity.com/paper/2111.01956/) (2021)
- [Parameter-Efficient Continual Fine-Tuning: A Survey](https://mlsanity.com/paper/2504.13822/) (2025)
- [Reactive Exploration to Cope with Non-Stationarity in Lifelong Reinforcement Learning](https://mlsanity.com/paper/2207.05742/) (2022)
- [Continual Feature Selection: Spurious Features in Continual Learning](https://mlsanity.com/paper/2203.01012/) (2022)
- [Learning towards Synchronous Network Memorizability and Generalizability for Continual Segmentation across Multiple Sites](https://mlsanity.com/paper/2206.06813/) (2022)
- [Online Continual Learning via the Meta-learning Update with Multi-scale Knowledge Distillation and Data Augmentation](https://mlsanity.com/paper/2209.06107/) (2022)
- [Continual Learning: Forget-free Winning Subnetworks for Video Representations](https://mlsanity.com/paper/2312.11973/) (2023)
- [TAG: Task-based Accumulated Gradients for Lifelong learning](https://mlsanity.com/paper/2105.05155/) (2021)
- [Forget-free Continual Learning with Soft-Winning SubNetworks](https://mlsanity.com/paper/2303.14962/) (2023)
- [Learning to Learn for Few-shot Continual Active Learning](https://mlsanity.com/paper/2311.03732/) (2023)
- [Fully Online Meta-Learning Without Task Boundaries](https://mlsanity.com/paper/2202.00263/) (2022)
- [Schedule-Robust Online Continual Learning](https://mlsanity.com/paper/2210.05561/) (2022)
- [HyperMask: Adaptive Hypernetwork-based Masks for Continual Learning](https://mlsanity.com/paper/2310.00113/) (2023)

## Abstract (arXiv)

The continual learning problem involves training models with limited capacity to perform well on a set of an unknown number of sequentially arriving tasks. While meta-learning shows great potential for reducing interference between old and new tasks, the current training procedures tend to be either slow or offline, and sensitive to many hyper-parameters. In this work, we propose Look-ahead MAML (La-MAML), a fast optimisation-based meta-learning algorithm for online-continual learning, aided by a small episodic memory. Our proposed modulation of per-parameter learning rates in our meta-learning update allows us to draw connections to prior work on hypergradients and meta-descent. This provides a more flexible and efficient way to mitigate catastrophic forgetting compared to conventional prior-based methods. La-MAML achieves performance superior to other replay-based, prior-based and meta-learning based approaches for continual learning on real-world visual classification benchmarks. Source code can be found here: https://github.com/montrealrobotics/La-MAML