mlsanityData to 7 Oct 2026

Beyond Supervised Continual Learning: a Review

arXiv 2208.14307 ↗First posted 30 Aug 2022cs.LGcited by 11

Not on the map. Why?

Abstract (arXiv)

Continual Learning (CL, sometimes also termed incremental learning) is a flavor of machine learning where the usual assumption of stationary data distribution is relaxed or omitted. When naively applying, e.g., DNNs in CL problems, changes in the data distribution can cause the so-called catastrophic forgetting (CF) effect: an abrupt loss of previous knowledge. Although many significant contributions to enabling CL have been made in recent years, most works address supervised (classification) problems. This article reviews literature that study CL in other settings, such as learning with reduced supervision, fully unsupervised learning, and reinforcement learning. Besides proposing a simple schema for classifying CL approaches w.r.t. their level of autonomy and supervision, we discuss the specific challenges associated with each setting and the potential contributions to the field of CL in general.

Key facts. Cited by 11 papers in the map (data through 7 Oct 2026). Builds on 98 of its 120 references that are in the map. First posted on arXiv 30 Aug 2022.

Builds on

98 of 120 references.

References · 98

  1. 2016Overcoming catastrophic forgetting in neural networkscited by 4,718
  2. 2013Playing Atari with Deep Reinforcement Learningcited by 3,211
  3. 2016iCaRL: Incremental Classifier and Representation Learningcited by 2,313
  4. 2017Gradient Episodic Memory for Continual Learningcited by 1,871
  5. 2016Progressive Neural Networkscited by 1,332
  6. 2019Meta-World: A Benchmark and Evaluation for Multi-Task and Meta Reinforcement Learningcited by 1,168
  7. 2017Continual Learning with Deep Generative Replaycited by 1,152
  8. 2017Hindsight Experience Replaycited by 1,096
  9. 2018Experience Replay for Continual Learningcited by 1,074
  10. 2015Prioritized Experience Replaycited by 1,070
  11. 2018Efficient Lifelong Learning with A-GEMcited by 1,044
  12. 2020Deep Reinforcement Learning for Autonomous Driving: A Surveycited by 698
38 more
  1. 2018Overcoming catastrophic forgetting with hard attention to the taskcited by 672
  2. 2019Three scenarios for continual learningcited by 610
  3. 2018Progress & Compress: A scalable framework for continual learningcited by 516
  4. 2017PathNet: Evolution Channels Gradient Descent in Super Neural Networkscited by 362
  5. 2019Orthogonal Gradient Descent for Continual Learningcited by 361
  6. 2017Measuring Catastrophic Forgetting in Neural Networkscited by 359
  7. 2020Towards Continual Reinforcement Learning: A Review and Perspectivescited by 268
  8. 2018Learning Not to Learn: Training Deep Neural Networks with Biased Datacited by 255
  9. 2018Selective Experience Replay for Lifelong Learningcited by 239
  10. 2017Distral: Robust Multitask Reinforcement Learningcited by 231
  11. 2021MT-Opt: Continuous Multi-Task Robotic Reinforcement Learning at Scalecited by 226
  12. 2020Supermasks in Superpositioncited by 223
  13. 2017Incremental Learning of Object Detectors without Catastrophic Forgettingcited by 208
  14. 2018Reinforced Continual Learningcited by 186
  15. 2020Revisiting Fundamentals of Experience Replaycited by 171
  16. 2019REPAIR: Removing Representation Bias by Dataset Resamplingcited by 157
  17. 2021Self-Supervised Models are Continual Learnerscited by 149
  18. 2019Continual Unsupervised Representation Learningcited by 147
  19. 2020Multi-Task Reinforcement Learning with Soft Modularizationcited by 138
  20. 2021Avalanche: an End-to-End Library for Continual Learningcited by 124
  21. 2019A continual learning survey: Defying forgetting in classification taskscited by 118
  22. 2018Don't forget, there is more than forgetting: new metrics for Continual Learningcited by 113
  23. 2021Continual World: A Robotic Benchmark For Continual Reinforcement Learningcited by 112
  24. 2018Continuous Learning in Single-Incremental-Task Scenarioscited by 109
  25. 2020A Comprehensive Study of Class Incremental Learning Algorithms for Visual Taskscited by 107
  26. 2017A Deeper Look at Experience Replaycited by 104
  27. 2020A Theoretical Analysis of Catastrophic Forgetting through the NTK Overlap Matrixcited by 99
  28. 2021Representational Continuity for Unsupervised Continual Learningcited by 95
  29. 2021Reset-Free Reinforcement Learning via Multi-Task Learning: Learning Dexterous Manipulation Behaviors without Human Interventioncited by 90
  30. 2020Efficient Continual Learning with Modular Networks and Task-Driven Priorscited by 87
  31. 2020A Wholistic View of Continual Learning with Deep Neural Networks: Forgotten Lessons and the Bridge to Active and Open World Learningcited by 86
  32. 2019Lifelong GAN: Continual Learning for Conditional Image Generationcited by 78
  33. 2020Imbalanced Continual Learning with Partitioning Reservoir Samplingcited by 71
  34. 2018Generative Models from the perspective of Continual Learningcited by 69
  35. 2020A Survey of Reinforcement Learning Algorithms for Dynamically Varying Environmentscited by 62
  36. 2018Task Agnostic Continual Learning Using Online Variational Bayescited by 61
  37. 2021Continual Learning via Local Module Compositioncited by 61
  38. 2020Removing Bias in Multi-modal Classifiers: Regularization by Maximizing Functional Entropiescited by 56

50 most cited of 98 All 98 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 3; 2024: 4; 2025: 3; 2026: 1.

11 papers.

  1. 2023Continual Learning as Computationally Constrained Reinforcement Learningcited by 35
  2. 2025Parameter-Efficient Continual Fine-Tuning: A Surveycited by 11
  3. 2023Evaluating Continual Learning on a Home Robotcited by 4
  4. 2023Adiabatic replay for continual learningcited by 3
  5. 2025Divide-and-Conquer for Enhancing Unlabeled Learning, Stability, and Plasticity in Semi-supervised Continual Learningcited by 3
  6. 2024CUAL: Continual Uncertainty-aware Active Learnercited by 1
  7. 2024Uncertainty Quantification in Continual Open-World Learningcited by 1
  8. 2024Deep evolving semi-supervised anomaly detectioncited by 0
  9. 2024CONCLAD: COntinuous Novel CLAss Detectorcited by 0
  10. 2025Demonstration-Guided Continual Reinforcement Learning in Dynamic Environmentscited by 0
  11. 2026Lifelong Representations: A Survey on Continual Self-Supervised Learning for Vision Modelscited by 0