# Theoretical Foundations of Continual Learning via Drift-Plus-Penalty

arXiv: https://arxiv.org/abs/2606.08452 · HTML: https://mlsanity.com/paper/2606.08452/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 28 of its 46 references that are in the map.
- First posted on arXiv 7 Jun 2026.

## Builds on (most cited first, 28 of 28)

- [Overcoming catastrophic forgetting in neural networks](https://mlsanity.com/paper/1612.00796/) (2016)
- [Learning without Forgetting](https://mlsanity.com/paper/1606.09282/) (2016)
- [Gradient Episodic Memory for Continual Learning](https://mlsanity.com/paper/1706.08840/) (2017)
- [Progressive Neural Networks](https://mlsanity.com/paper/1606.04671/) (2016)
- [On Large-Batch Training for Deep Learning: Generalization Gap and Sharp Minima](https://mlsanity.com/paper/1609.04836/) (2016)
- [Memory Aware Synapses: Learning what (not) to forget](https://mlsanity.com/paper/1711.09601/) (2017)
- [Experience Replay for Continual Learning](https://mlsanity.com/paper/1811.11682/) (2018)
- [Efficient Lifelong Learning with A-GEM](https://mlsanity.com/paper/1812.00420/) (2018)
- [Dark Experience for General Continual Learning: a Strong, Simple Baseline](https://mlsanity.com/paper/2004.07211/) (2020)
- [Learning to Learn without Forgetting by Maximizing Transfer and Minimizing Interference](https://mlsanity.com/paper/1810.11910/) (2018)
- [Online Learning: A Modern Introduction Using Convex Optimization](https://mlsanity.com/paper/1912.13213/) (2019)
- [Orthogonal Gradient Descent for Continual Learning](https://mlsanity.com/paper/1910.07104/) (2019)
- [Loss landscapes and optimization in over-parameterized non-linear systems and neural networks](https://mlsanity.com/paper/2003.00307/) (2020)
- [New Insights on Reducing Abrupt Representation Change in Online Continual Learning](https://mlsanity.com/paper/2104.05025/) (2021)
- [Avalanche: an End-to-End Library for Continual Learning](https://mlsanity.com/paper/2104.00405/) (2021)
- [Theory on Forgetting and Generalization of Continual Learning](https://mlsanity.com/paper/2302.05836/) (2023)
- [Beyond Not-Forgetting: Continual Learning with Backward Knowledge Transfer](https://mlsanity.com/paper/2211.00789/) (2022)
- [Optimal Algorithms for Online Convex Optimization with Adversarial Constraints](https://mlsanity.com/paper/2310.18955/) (2023)
- [Continual Learning with Recursive Gradient Optimization](https://mlsanity.com/paper/2201.12522/) (2022)
- [A Unified and General Framework for Continual Learning](https://mlsanity.com/paper/2403.13249/) (2024)
- [CBA: Improving Online Continual Learning via Continual Bias Adaptor](https://mlsanity.com/paper/2308.06925/) (2023)
- [Projection-free Algorithms for Online Convex Optimization with Adversarial Constraints](https://mlsanity.com/paper/2501.16919/) (2025)
- [Online Curvature-Aware Replay: Leveraging mathbf2nd Order Information for Online Continual Learning](https://mlsanity.com/paper/2502.01866/) (2025)
- [DRAE: Dynamic Retrieval-Augmented Expert Networks for Lifelong Learning and Task Adaptation in Robotics](https://mlsanity.com/paper/2507.04661/) (2025)
- [On the Convergence of Continual Learning with Adaptive Methods](https://mlsanity.com/paper/2404.05555/) (2024)
- [CORE: Mitigating Catastrophic Forgetting in Continual Learning through Cognitive Replay](https://mlsanity.com/paper/2402.01348/) (2024)
- [On the Convergence of Continual Federated Learning Using Incrementally Aggregated Gradients](https://mlsanity.com/paper/2411.07959/) (2024)
- [A Lyapunov Drift-Plus-Penalty Method Tailored for Reinforcement Learning with Queue Stability](https://mlsanity.com/paper/2506.04291/) (2025)

## Built on by (most cited first, 1 of 1)

- [Federated Continual Learning as a Distributed Drift-Plus-Penalty Control Problem](https://mlsanity.com/paper/2608.21539/) (2026)

## Abstract (arXiv)

In many real-world settings, data streams are nonstationary and arrive sequentially, requiring learning systems to adapt continuously without retraining from scratch. Continual learning (CL) addresses this challenge by incorporating new tasks while mitigating catastrophic forgetting, where learning new information degrades performance on previously acquired knowledge. We introduce a control-theoretic perspective on CL that explicitly regulates the evolution of forgetting, framing adaptation as a controlled process subject to long-term stability constraints. We focus on replay-based CL, where a finite memory buffer stores representative samples from prior tasks. We propose COntinual Learning with Drift-Plus-Penalty (COLD), a continual learning framework based on the Drift-Plus-Penalty (DPP) principle from stochastic optimization. To facilitate analysis, we also consider an oracle variant, COLD-ORACLE, as a reference benchmark. At each task, both methods minimize the current task loss while maintaining a virtual queue that tracks deviations from long-term stability on previously learned tasks, capturing the stability-plasticity trade-off as a regulated dynamical process. We establish stability and convergence guarantees that characterize this trade-off through a tunable control parameter. Experiments on standard benchmarks demonstrate that COLD consistently outperforms a broad range of state-of-the-art CL methods while providing competitive and controllable forgetting behavior through explicit regulation of stability and plasticity.