# The Ideal Continual Learner: An Agent That Never Forgets

arXiv: https://arxiv.org/abs/2305.00316 · HTML: https://mlsanity.com/paper/2305.00316/

## Key facts

- Cited by 35 papers in the map (data through 14 Oct 2026).
- Builds on 87 of its 145 references that are in the map.
- First posted on arXiv 29 Apr 2023.

## Builds on (most cited first, 30 of 87)

- [Overcoming catastrophic forgetting in neural networks](https://mlsanity.com/paper/1612.00796/) (2016)
- [iCaRL: Incremental Classifier and Representation Learning](https://mlsanity.com/paper/1611.07725/) (2016)
- [Gradient Episodic Memory for Continual Learning](https://mlsanity.com/paper/1706.08840/) (2017)
- [Theoretically Principled Trade-off between Robustness and Accuracy](https://mlsanity.com/paper/1901.08573/) (2019)
- [Progressive Neural Networks](https://mlsanity.com/paper/1606.04671/) (2016)
- [Continual Lifelong Learning with Neural Networks: A Review](https://mlsanity.com/paper/1802.07569/) (2018)
- [Continual Learning with Deep Generative Replay](https://mlsanity.com/paper/1705.08690/) (2017)
- [Memory Aware Synapses: Learning what (not) to forget](https://mlsanity.com/paper/1711.09601/) (2017)
- [Dark Experience for General Continual Learning: a Strong, Simple Baseline](https://mlsanity.com/paper/2004.07211/) (2020)
- [PackNet: Adding Multiple Tasks to a Single Network by Iterative Pruning](https://mlsanity.com/paper/1711.05769/) (2017)
- [Lifelong Learning with Dynamically Expandable Networks](https://mlsanity.com/paper/1708.01547/) (2017)
- [Gradient based sample selection for online continual learning](https://mlsanity.com/paper/1903.08671/) (2019)
- [Learning to Learn without Forgetting by Maximizing Transfer and Minimizing Interference](https://mlsanity.com/paper/1810.11910/) (2018)
- [Orthogonal Gradient Descent for Continual Learning](https://mlsanity.com/paper/1910.07104/) (2019)
- [DER: Dynamically Expandable Representation for Class Incremental Learning](https://mlsanity.com/paper/2103.16788/) (2021)
- [Gradient Projection Memory for Continual Learning](https://mlsanity.com/paper/2103.09762/) (2021)
- [Learn to Grow: A Continual Structure Learning Framework for Overcoming Catastrophic Forgetting](https://mlsanity.com/paper/1904.00310/) (2019)
- [Efficient and Accurate Estimation of Lipschitz Constants for Deep Neural Networks](https://mlsanity.com/paper/1906.04893/) (2019)
- [DyTox: Transformers for Continual Learning with DYnamic TOken eXpansion](https://mlsanity.com/paper/2111.11326/) (2021)
- [Rainbow Memory: Continual Learning with a Memory of Diverse Samples](https://mlsanity.com/paper/2103.17230/) (2021)
- [Selective Experience Replay for Lifelong Learning](https://mlsanity.com/paper/1802.10269/) (2018)
- [Few-Shot Learning via Learning the Representation, Provably](https://mlsanity.com/paper/2002.09434/) (2020)
- [Compacting, Picking and Growing for Unforgetting Continual Learning](https://mlsanity.com/paper/1910.06562/) (2019)
- [Online Structured Laplace Approximations For Overcoming Catastrophic Forgetting](https://mlsanity.com/paper/1805.07810/) (2018)
- [The Benefit of Multitask Representation Learning](https://mlsanity.com/paper/1505.06279/) (2015)
- [Coresets via Bilevel Optimization for Continual Learning and Streaming](https://mlsanity.com/paper/2006.03875/) (2020)
- [Continual Learning of Context-dependent Processing in Neural Networks](https://mlsanity.com/paper/1810.01256/) (2018)
- [Constrained Reinforcement Learning Has Zero Duality Gap](https://mlsanity.com/paper/1910.13393/) (2019)
- [Linear Mode Connectivity in Multitask and Continual Learning](https://mlsanity.com/paper/2010.04495/) (2020)
- [Training Networks in Null Space of Feature Covariance for Continual Learning](https://mlsanity.com/paper/2103.07113/) (2021)

## Built on by (most cited first, 30 of 35)

- [The Emerged Security and Privacy of LLM Agent: A Survey with Case Studies](https://mlsanity.com/paper/2407.19354/) (2024)
- [A Comprehensive Survey of Forgetting in Deep Learning Beyond Continual Learning](https://mlsanity.com/paper/2307.09218/) (2023)
- [Theory on Mixture-of-Experts in Continual Learning](https://mlsanity.com/paper/2406.16437/) (2024)
- [Continual Learning in Linear Classification on Separable Data](https://mlsanity.com/paper/2306.03534/) (2023)
- [Resilient Constrained Learning](https://mlsanity.com/paper/2306.02426/) (2023)
- [Optimal Task Order for Continual Learning of Multiple Tasks](https://mlsanity.com/paper/2502.03350/) (2025)
- [Skill Expansion and Composition in Parameter Space](https://mlsanity.com/paper/2502.05932/) (2025)
- [On the Diminishing Returns of Width for Continual Learning](https://mlsanity.com/paper/2403.06398/) (2024)
- [From Continual Learning to SGD and Back: Better Rates for Continual Linear Models](https://mlsanity.com/paper/2504.04579/) (2025)
- [Fast Last-Iterate Convergence of SGD in the Smooth Interpolation Regime](https://mlsanity.com/paper/2507.11274/) (2025)
- [Last Iterate Convergence of Incremental Methods and Applications in Continual Learning](https://mlsanity.com/paper/2403.06873/) (2024)
- [LoRanPAC: Low-rank Random Features and Pre-trained Models for Bridging Theory and Practice in Continual Learning](https://mlsanity.com/paper/2410.00645/) (2024)
- [Optimal Rates in Continual Linear Regression via Increasing Regularization](https://mlsanity.com/paper/2506.06501/) (2025)
- [Continual Reinforcement Learning by Planning with Online World Models](https://mlsanity.com/paper/2507.09177/) (2025)
- [Solving Continual Offline RL through Selective Weights Activation on Aligned Spaces](https://mlsanity.com/paper/2410.15698/) (2024)
- [Measuring Representational Shifts in Continual Learning: A Linear Transformation Perspective](https://mlsanity.com/paper/2505.20970/) (2025)
- [Replay Can Provably Increase Forgetting](https://mlsanity.com/paper/2506.04377/) (2025)
- [Primal Dual Continual Learning: Balancing Stability and Plasticity through Adaptive Memory Allocation](https://mlsanity.com/paper/2310.00154/) (2023)
- [A Retention-Centric Framework for Continual Learning with Guaranteed Model Developmental Safety](https://mlsanity.com/paper/2410.03955/) (2024)
- [Prospective Learning: Learning for a Dynamic Future](https://mlsanity.com/paper/2411.00109/) (2024)
- [Enhancing Pretrained Model-based Continual Representation Learning via Guided Random Projection](https://mlsanity.com/paper/2603.19145/) (2026)
- [Studying Generalization on Memory-Based Methods in Continual Learning](https://mlsanity.com/paper/2306.09890/) (2023)
- [Global Convergence of Continual Learning on Non-IID Data](https://mlsanity.com/paper/2503.18511/) (2025)
- [One Rank at a Time: Cascading Error Dynamics in Sequential Learning](https://mlsanity.com/paper/2505.22602/) (2025)
- [Continual Diffuser (CoD): Mastering Continual Offline Reinforcement Learning with Experience Rehearsal](https://mlsanity.com/paper/2409.02512/) (2024)
- [Parabolic Continual Learning](https://mlsanity.com/paper/2503.02117/) (2025)
- [Analysis of the Memorization and Generalization Capabilities of AI Agents: Are Continual Learners Robust?](https://mlsanity.com/paper/2309.10149/) (2023)
- [Recovery Guarantees for Continual Learning of Dependent Tasks: Memory, Data-Dependent Regularization, and Data-Dependent Weights](https://mlsanity.com/paper/2604.17578/) (2026)
- [Mathematics of Continual Learning](https://mlsanity.com/paper/2504.17963/) (2025)
- [Understanding the Role of Rehearsal Scale in Continual Learning under Varying Model Capacities](https://mlsanity.com/paper/2602.20791/) (2026)

## Abstract (arXiv)

The goal of continual learning is to find a model that solves multiple learning tasks which are presented sequentially to the learner. A key challenge in this setting is that the learner may forget how to solve a previous task when learning a new task, a phenomenon known as catastrophic forgetting. To address this challenge, many practical methods have been proposed, including memory-based, regularization-based, and expansion-based methods. However, a rigorous theoretical understanding of these methods remains elusive. This paper aims to bridge this gap between theory and practice by proposing a new continual learning framework called Ideal Continual Learner (ICL), which is guaranteed to avoid catastrophic forgetting by construction. We show that ICL unifies multiple well-established continual learning methods and gives new theoretical insights into the strengths and weaknesses of these methods. We also derive generalization bounds for ICL which allow us to theoretically quantify how rehearsal affects generalization. Finally, we connect ICL to several classic subjects and research topics of modern interest, which allows us to make historical remarks and inspire future directions.