# Layerwise Proximal Replay: A Proximal Point Method for Online Continual Learning

arXiv: https://arxiv.org/abs/2402.09542 · HTML: https://mlsanity.com/paper/2402.09542/

## Key facts

- Cited by 13 papers in the map (data through 14 Oct 2026).
- Builds on 24 of its 42 references that are in the map.
- First posted on arXiv 14 Feb 2024.

## Builds on (most cited first, 24 of 24)

- [Efficient Lifelong Learning with A-GEM](https://mlsanity.com/paper/1812.00420/) (2018)
- [Dark Experience for General Continual Learning: a Strong, Simple Baseline](https://mlsanity.com/paper/2004.07211/) (2020)
- [Optimizing Neural Networks with Kronecker-factored Approximate Curvature](https://mlsanity.com/paper/1503.05671/) (2015)
- [Orthogonal Gradient Descent for Continual Learning](https://mlsanity.com/paper/1910.07104/) (2019)
- [Gradient Projection Memory for Continual Learning](https://mlsanity.com/paper/2103.09762/) (2021)
- [On Tiny Episodic Memories in Continual Learning](https://mlsanity.com/paper/1902.10486/) (2019)
- [CORe50: a New Dataset and Benchmark for Continuous Object Recognition](https://mlsanity.com/paper/1705.03550/) (2017)
- [New Insights on Reducing Abrupt Representation Change in Online Continual Learning](https://mlsanity.com/paper/2104.05025/) (2021)
- [Continual Learning of Context-dependent Processing in Neural Networks](https://mlsanity.com/paper/1810.01256/) (2018)
- [The CLEAR Benchmark: Continual LEArning on Real-World Imagery](https://mlsanity.com/paper/2201.06289/) (2022)
- [TRGP: Trust Region Gradient Projection for Continual Learning](https://mlsanity.com/paper/2202.02931/) (2022)
- [Flattening Sharpness for Dynamic Gradient Projection Memory Benefits Continual Learning](https://mlsanity.com/paper/2110.04593/) (2021)
- [Stochastic (Approximate) Proximal Point Methods: Convergence, Optimality, and Adaptivity](https://mlsanity.com/paper/1810.05633/) (2018)
- [Continual evaluation for lifelong learning: Identifying the stability gap](https://mlsanity.com/paper/2205.13452/) (2022)
- [Natural continual learning: success is a journey, not (just) a destination](https://mlsanity.com/paper/2106.08085/) (2021)
- [Parameter-Level Soft-Masking for Continual Learning](https://mlsanity.com/paper/2306.14775/) (2023)
- [Online Fast Adaptation and Knowledge Accumulation: a New Approach to Continual Learning](https://mlsanity.com/paper/2003.05856/) (2020)
- [Continual Learning with Scaled Gradient Projection](https://mlsanity.com/paper/2302.01386/) (2023)
- [Avalanche: A PyTorch Library for Deep Continual Learning](https://mlsanity.com/paper/2302.01766/) (2023)
- [A simple but strong baseline for online continual learning: Repeated Augmented Rehearsal](https://mlsanity.com/paper/2209.13917/) (2022)
- [Two Complementary Perspectives to Continual Learning: Ask Not Only What to Optimize, But Also How](https://mlsanity.com/paper/2311.04898/) (2023)
- [SPACE: Structured Compression and Sharing of Representational Space for Continual Learning](https://mlsanity.com/paper/2001.08650/) (2020)
- [Improving Online Continual Learning Performance and Stability with Temporal Ensembles](https://mlsanity.com/paper/2306.16817/) (2023)
- [The proximal point method revisited](https://mlsanity.com/paper/1712.06038/) (2017)

## Built on by (most cited first, 13 of 13)

- [Online Curvature-Aware Replay: Leveraging mathbf2nd Order Information for Online Continual Learning](https://mlsanity.com/paper/2502.01866/) (2025)
- [Region-Based Optimization in Continual Learning for Audio Deepfake Detection](https://mlsanity.com/paper/2412.11551/) (2024)
- [Lifelong Learning of Video Diffusion Models From a Single Video Stream](https://mlsanity.com/paper/2406.04814/) (2024)
- [Improvements of Dark Experience Replay and Reservoir Sampling towards Better Balance between Consolidation and Plasticity](https://mlsanity.com/paper/2504.20932/) (2025)
- [PROL : Rehearsal Free Continual Learning in Streaming Data via Prompt Online Learning](https://mlsanity.com/paper/2507.12305/) (2025)
- [C-Flat++: Towards a More Efficient and Powerful Framework for Continual Learning](https://mlsanity.com/paper/2508.18860/) (2025)
- [Towards Robust Incremental Learning under Ambiguous Supervision](https://mlsanity.com/paper/2501.13584/) (2025)
- [Online Continual Learning for Time Series: a Natural Score-driven Approach](https://mlsanity.com/paper/2601.12931/) (2026)
- [Finding Structure in Continual Learning](https://mlsanity.com/paper/2602.04555/) (2026)
- [Unsupervised Continual Learning for Amortized Bayesian Inference](https://mlsanity.com/paper/2602.22884/) (2026)
- [Memory-efficient Continual Learning with Prototypical Exemplar Condensation](https://mlsanity.com/paper/2603.13804/) (2026)
- [Task Switching Without Forgetting via Proximal Decoupling](https://mlsanity.com/paper/2604.18857/) (2026)
- [Formalizing Forgetting: Theory of Forgetting for Online Continual Learning](https://mlsanity.com/paper/2610.14787/) (2026)

## Abstract (arXiv)

In online continual learning, a neural network incrementally learns from a non-i.i.d. data stream. Nearly all online continual learning methods employ experience replay to simultaneously prevent catastrophic forgetting and underfitting on past data. Our work demonstrates a limitation of this approach: neural networks trained with experience replay tend to have unstable optimization trajectories, impeding their overall accuracy. Surprisingly, these instabilities persist even when the replay buffer stores all previous training examples, suggesting that this issue is orthogonal to catastrophic forgetting. We minimize these instabilities through a simple modification of the optimization geometry. Our solution, Layerwise Proximal Replay (LPR), balances learning from new and replay data while only allowing for gradual changes in the hidden activation of past data. We demonstrate that LPR consistently improves replay-based online continual learning methods across multiple problem settings, regardless of the amount of available replay memory.