# Sequential mastery of multiple visual tasks: Networks naturally learn to learn and forget to forget

arXiv: https://arxiv.org/abs/1905.10837 · HTML: https://mlsanity.com/paper/1905.10837/

## Key facts

- Cited by 13 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 26 May 2019.

## Built on by (most cited first, 13 of 13)

- [Measuring Forgetting of Memorized Training Examples](https://mlsanity.com/paper/2207.00099/) (2022)
- [Replay in Deep Learning: Current Approaches and Missing Biological Elements](https://mlsanity.com/paper/2104.04132/) (2021)
- [RMM: Reinforced Memory Management for Class-Incremental Learning](https://mlsanity.com/paper/2301.05792/) (2023)
- [Factors of Influence for Transfer Learning across Diverse Appearance Domains and Task Types](https://mlsanity.com/paper/2103.13318/) (2021)
- [Incremental Learning Meets Transfer Learning: Application to Multi-site Prostate MRI Segmentation](https://mlsanity.com/paper/2206.01369/) (2022)
- [Discrete Key-Value Bottleneck](https://mlsanity.com/paper/2207.11240/) (2022)
- [A comprehensive and easy-to-use multi-domain multi-task medical imaging meta-dataset](https://mlsanity.com/paper/2404.16000/) (2024)
- [Reawakening knowledge: Anticipatory recovery from catastrophic interference via structured training](https://mlsanity.com/paper/2403.09613/) (2024)
- [On robustness of generative representations against catastrophic forgetting](https://mlsanity.com/paper/2109.01844/) (2021)
- [Thalamus: a brain-inspired algorithm for biologically-plausible continual learning and disentangled representations](https://mlsanity.com/paper/2205.11713/) (2022)
- [Natural Mitigation of Catastrophic Interference: Continual Learning in Power-Law Learning Environments](https://mlsanity.com/paper/2401.10393/) (2024)
- [Bridging Neuroscience and AI: Environmental Enrichment as a Model for Forward Knowledge Transfer](https://mlsanity.com/paper/2405.07295/) (2024)
- [DILF-EN framework for Class-Incremental Learning](https://mlsanity.com/paper/2112.12385/) (2021)

## Abstract (arXiv)

We explore the behavior of a standard convolutional neural net in a continual-learning setting that introduces visual classification tasks sequentially and requires the net to master new tasks while preserving mastery of previously learned tasks. This setting corresponds to that which human learners face as they acquire domain expertise serially, for example, as an individual studies a textbook. Through simulations involving sequences of ten related visual tasks, we find reason for optimism that nets will scale well as they advance from having a single skill to becoming multi-skill domain experts. We observe two key phenomena. First, forward facilitation---the accelerated learning of task n+1 having learned n previous tasks---grows with n. Second, backward interference---the forgetting of the n previous tasks when learning task n+1---diminishes with n. Amplifying forward facilitation is the goal of research on metalearning, and attenuating backward interference is the goal of research on catastrophic forgetting. We find that both of these goals are attained simply through broader exposure to a domain.