# Beyond Supervised Continual Learning: a Review

arXiv: https://arxiv.org/abs/2208.14307 · HTML: https://mlsanity.com/paper/2208.14307/

## Key facts

- Cited by 11 papers in the map (data through 14 Oct 2026).
- Builds on 98 of its 120 references that are in the map.
- First posted on arXiv 30 Aug 2022.

## Builds on (most cited first, 30 of 98)

- [Overcoming catastrophic forgetting in neural networks](https://mlsanity.com/paper/1612.00796/) (2016)
- [Playing Atari with Deep Reinforcement Learning](https://mlsanity.com/paper/1312.5602/) (2013)
- [iCaRL: Incremental Classifier and Representation Learning](https://mlsanity.com/paper/1611.07725/) (2016)
- [Gradient Episodic Memory for Continual Learning](https://mlsanity.com/paper/1706.08840/) (2017)
- [Progressive Neural Networks](https://mlsanity.com/paper/1606.04671/) (2016)
- [Meta-World: A Benchmark and Evaluation for Multi-Task and Meta Reinforcement Learning](https://mlsanity.com/paper/1910.10897/) (2019)
- [Continual Learning with Deep Generative Replay](https://mlsanity.com/paper/1705.08690/) (2017)
- [Hindsight Experience Replay](https://mlsanity.com/paper/1707.01495/) (2017)
- [Prioritized Experience Replay](https://mlsanity.com/paper/1511.05952/) (2015)
- [Experience Replay for Continual Learning](https://mlsanity.com/paper/1811.11682/) (2018)
- [Efficient Lifelong Learning with A-GEM](https://mlsanity.com/paper/1812.00420/) (2018)
- [Deep Reinforcement Learning for Autonomous Driving: A Survey](https://mlsanity.com/paper/2002.00444/) (2020)
- [Overcoming catastrophic forgetting with hard attention to the task](https://mlsanity.com/paper/1801.01423/) (2018)
- [Three scenarios for continual learning](https://mlsanity.com/paper/1904.07734/) (2019)
- [Progress & Compress: A scalable framework for continual learning](https://mlsanity.com/paper/1805.06370/) (2018)
- [Orthogonal Gradient Descent for Continual Learning](https://mlsanity.com/paper/1910.07104/) (2019)
- [PathNet: Evolution Channels Gradient Descent in Super Neural Networks](https://mlsanity.com/paper/1701.08734/) (2017)
- [Measuring Catastrophic Forgetting in Neural Networks](https://mlsanity.com/paper/1708.02072/) (2017)
- [Towards Continual Reinforcement Learning: A Review and Perspectives](https://mlsanity.com/paper/2012.13490/) (2020)
- [Learning Not to Learn: Training Deep Neural Networks with Biased Data](https://mlsanity.com/paper/1812.10352/) (2018)
- [Selective Experience Replay for Lifelong Learning](https://mlsanity.com/paper/1802.10269/) (2018)
- [Distral: Robust Multitask Reinforcement Learning](https://mlsanity.com/paper/1707.04175/) (2017)
- [MT-Opt: Continuous Multi-Task Robotic Reinforcement Learning at Scale](https://mlsanity.com/paper/2104.08212/) (2021)
- [Supermasks in Superposition](https://mlsanity.com/paper/2006.14769/) (2020)
- [Incremental Learning of Object Detectors without Catastrophic Forgetting](https://mlsanity.com/paper/1708.06977/) (2017)
- [Reinforced Continual Learning](https://mlsanity.com/paper/1805.12369/) (2018)
- [Revisiting Fundamentals of Experience Replay](https://mlsanity.com/paper/2007.06700/) (2020)
- [REPAIR: Removing Representation Bias by Dataset Resampling](https://mlsanity.com/paper/1904.07911/) (2019)
- [Self-Supervised Models are Continual Learners](https://mlsanity.com/paper/2112.04215/) (2021)
- [Continual Unsupervised Representation Learning](https://mlsanity.com/paper/1910.14481/) (2019)

## Built on by (most cited first, 11 of 11)

- [Continual Learning as Computationally Constrained Reinforcement Learning](https://mlsanity.com/paper/2307.04345/) (2023)
- [Parameter-Efficient Continual Fine-Tuning: A Survey](https://mlsanity.com/paper/2504.13822/) (2025)
- [Evaluating Continual Learning on a Home Robot](https://mlsanity.com/paper/2306.02413/) (2023)
- [Adiabatic replay for continual learning](https://mlsanity.com/paper/2303.13157/) (2023)
- [Divide-and-Conquer for Enhancing Unlabeled Learning, Stability, and Plasticity in Semi-supervised Continual Learning](https://mlsanity.com/paper/2508.05316/) (2025)
- [CUAL: Continual Uncertainty-aware Active Learner](https://mlsanity.com/paper/2412.09701/) (2024)
- [Uncertainty Quantification in Continual Open-World Learning](https://mlsanity.com/paper/2412.16409/) (2024)
- [Deep evolving semi-supervised anomaly detection](https://mlsanity.com/paper/2412.00860/) (2024)
- [CONCLAD: COntinuous Novel CLAss Detector](https://mlsanity.com/paper/2412.10473/) (2024)
- [Demonstration-Guided Continual Reinforcement Learning in Dynamic Environments](https://mlsanity.com/paper/2512.18670/) (2025)
- [Lifelong Representations: A Survey on Continual Self-Supervised Learning for Vision Models](https://mlsanity.com/paper/2607.09785/) (2026)

## Abstract (arXiv)

Continual Learning (CL, sometimes also termed incremental learning) is a flavor of machine learning where the usual assumption of stationary data distribution is relaxed or omitted. When naively applying, e.g., DNNs in CL problems, changes in the data distribution can cause the so-called catastrophic forgetting (CF) effect: an abrupt loss of previous knowledge. Although many significant contributions to enabling CL have been made in recent years, most works address supervised (classification) problems. This article reviews literature that study CL in other settings, such as learning with reduced supervision, fully unsupervised learning, and reinforcement learning. Besides proposing a simple schema for classifying CL approaches w.r.t. their level of autonomy and supervision, we discuss the specific challenges associated with each setting and the potential contributions to the field of CL in general.