# Meta-learnt priors slow down catastrophic forgetting in neural networks

arXiv: https://arxiv.org/abs/1909.04170 · HTML: https://mlsanity.com/paper/1909.04170/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 9 Sep 2019.

## Built on by (most cited first, 5 of 5)

- [Building a Subspace of Policies for Scalable Continual Learning](https://mlsanity.com/paper/2211.10445/) (2022)
- [Meta-Learning with Neural Tangent Kernels](https://mlsanity.com/paper/2102.03909/) (2021)
- [Generalising via Meta-Examples for Continual Learning in the Wild](https://mlsanity.com/paper/2101.12081/) (2021)
- [Reviewing continual learning from the perspective of human-level intelligence](https://mlsanity.com/paper/2111.11964/) (2021)
- [Catastrophic Forgetting in Kolmogorov-Arnold Networks](https://mlsanity.com/paper/2511.12828/) (2025)

## Abstract (arXiv)

Current training regimes for deep learning usually involve exposure to a single task / dataset at a time. Here we start from the observation that in this context the trained model is not given any knowledge of anything outside its (single-task) training distribution, and has thus no way to learn parameters (i.e., feature detectors or policies) that could be helpful to solve other tasks, and to limit future interference with the acquired knowledge, and thus catastrophic forgetting. Here we show that catastrophic forgetting can be mitigated in a meta-learning context, by exposing a neural network to multiple tasks in a sequential manner during training. Finally, we present SeqFOMAML, a meta-learning algorithm that implements these principles, and we evaluate it on sequential learning problems composed by Omniglot and MiniImageNet classification tasks.