# Maintaining Plasticity in Continual Learning via Regenerative Regularization

arXiv: https://arxiv.org/abs/2308.11958 · HTML: https://mlsanity.com/paper/2308.11958/

## Key facts

- Cited by 65 papers in the map (data through 14 Oct 2026).
- Builds on 24 of its 30 references that are in the map.
- First posted on arXiv 23 Aug 2023.

## Builds on (most cited first, 24 of 24)

- [Layer Normalization](https://mlsanity.com/paper/1607.06450/) (2016)
- [Overcoming catastrophic forgetting in neural networks](https://mlsanity.com/paper/1612.00796/) (2016)
- [Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification](https://mlsanity.com/paper/1502.01852/) (2015)
- [Playing Atari with Deep Reinforcement Learning](https://mlsanity.com/paper/1312.5602/) (2013)
- [Gradient Episodic Memory for Continual Learning](https://mlsanity.com/paper/1706.08840/) (2017)
- [Riemannian Walk for Incremental Learning: Understanding Forgetting and Intransigence](https://mlsanity.com/paper/1801.10112/) (2018)
- [An Empirical Investigation of Catastrophic Forgetting in Gradient-Based Neural Networks](https://mlsanity.com/paper/1312.6211/) (2013)
- [The Primacy Bias in Deep Reinforcement Learning](https://mlsanity.com/paper/2205.07802/) (2022)
- [The Dormant Neuron Phenomenon in Deep Reinforcement Learning](https://mlsanity.com/paper/2302.12902/) (2023)
- [Understanding plasticity in neural networks](https://mlsanity.com/paper/2303.01486/) (2023)
- [Implicit Under-Parameterization Inhibits Data-Efficient Deep Reinforcement Learning](https://mlsanity.com/paper/2010.14498/) (2020)
- [Understanding and Preventing Capacity Loss in Reinforcement Learning](https://mlsanity.com/paper/2204.09560/) (2022)
- [Loss of Plasticity in Continual Deep Reinforcement Learning](https://mlsanity.com/paper/2303.07507/) (2023)
- [Understanding and Improving Convolutional Neural Networks via Concatenated Rectified Linear Units](https://mlsanity.com/paper/1603.05201/) (2016)
- [Continual Backprop: Stochastic Gradient Descent with Persistent Randomness](https://mlsanity.com/paper/2108.06325/) (2021)
- [Critical Learning Periods in Deep Neural Networks](https://mlsanity.com/paper/1711.08856/) (2017)
- [Transient Non-Stationarity and Generalisation in Deep Reinforcement Learning](https://mlsanity.com/paper/2006.05826/) (2020)
- [Deep Reinforcement Learning with Plasticity Injection](https://mlsanity.com/paper/2305.15555/) (2023)
- [Online Continual Learning with Natural Distribution Shifts: An Empirical Study with Visual Data](https://mlsanity.com/paper/2108.09020/) (2021)
- [Bad Global Minima Exist and SGD Can Reach Them](https://mlsanity.com/paper/1906.02613/) (2019)
- [Maintaining Plasticity in Deep Continual Learning](https://mlsanity.com/paper/2306.13812/) (2023)
- [Online Continual Learning Without the Storage Constraint](https://mlsanity.com/paper/2305.09253/) (2023)
- [An Empirical Study of Implicit Regularization in Deep Offline RL](https://mlsanity.com/paper/2207.02099/) (2022)
- [Does the Adam Optimizer Exacerbate Catastrophic Forgetting?](https://mlsanity.com/paper/2102.07686/) (2021)

## Built on by (most cited first, 30 of 65)

- [Overestimation, Overfitting, and Plasticity in Actor-Critic: the Bitter Lesson of Reinforcement Learning](https://mlsanity.com/paper/2403.00514/) (2024)
- [No Representation, No Trust: Connecting Representation, Collapse, and Trust Issues in PPO](https://mlsanity.com/paper/2405.00662/) (2024)
- [SFT Doesn't Always Hurt General Capabilities: Revisiting Domain-Specific Fine-Tuning in LLMs](https://mlsanity.com/paper/2509.20758/) (2025)
- [Mitigating Plasticity Loss in Continual Reinforcement Learning by Reducing Churn](https://mlsanity.com/paper/2506.00592/) (2025)
- [Non-Stationary Learning of Neural Networks with Automatic Soft Parameter Reset](https://mlsanity.com/paper/2411.04034/) (2024)
- [Optimizers Qualitatively Alter Solutions And We Should Leverage This](https://mlsanity.com/paper/2507.12224/) (2025)
- [Measure gradients, not activations! Enhancing neuronal activity in deep reinforcement learning](https://mlsanity.com/paper/2505.24061/) (2025)
- [Rethinking Safety in LLM Fine-tuning: An Optimization Perspective](https://mlsanity.com/paper/2508.12531/) (2025)
- [Simplicial Embeddings Improve Sample Efficiency in Actor-Critic Agents](https://mlsanity.com/paper/2510.13704/) (2025)
- [Position: Lifetime tuning is incompatible with continual reinforcement learning](https://mlsanity.com/paper/2404.02113/) (2024)
- [A Forget-and-Grow Strategy for Deep Reinforcement Learning Scaling in Continuous Control](https://mlsanity.com/paper/2507.02712/) (2025)
- [FIRE: Frobenius-Isometry Reinitialization for Balancing the Stability-Plasticity Tradeoff](https://mlsanity.com/paper/2602.08040/) (2026)
- [Reinitializing weights vs units for maintaining plasticity in neural networks](https://mlsanity.com/paper/2508.00212/) (2025)
- [Activation Function Design Sustains Plasticity in Continual Learning](https://mlsanity.com/paper/2509.22562/) (2025)
- [Context-Free Synthetic Data Mitigates Forgetting](https://mlsanity.com/paper/2505.13811/) (2025)
- [Barriers for Learning in an Evolving World: Mathematical Understanding of Loss of Plasticity](https://mlsanity.com/paper/2510.00304/) (2025)
- [Same accuracy, twice as fast: continuous training surpasses retraining from scratch](https://mlsanity.com/paper/2502.21147/) (2025)
- [Variance Control via Weight Rescaling in LLM Pre-training](https://mlsanity.com/paper/2503.17500/) (2025)
- [Experience Replay Addresses Loss of Plasticity in Continual Learning](https://mlsanity.com/paper/2503.20018/) (2025)
- [Optimistic critics can empower small actors](https://mlsanity.com/paper/2506.01016/) (2025)
- [Non-Uniform Noise-to-Signal Ratio in the REINFORCE Policy-Gradient Estimator](https://mlsanity.com/paper/2602.01460/) (2026)
- [Weight Decay Improves Language Model Plasticity](https://mlsanity.com/paper/2602.11137/) (2026)
- [Predicting Plasticity in Deep Continual Learning: A Theoretical Perspective](https://mlsanity.com/paper/2605.09044/) (2026)
- [Rotation-Preserving Supervised Fine-Tuning](https://mlsanity.com/paper/2605.10973/) (2026)
- [Hierarchical Subspaces of Policies for Continual Offline Reinforcement Learning](https://mlsanity.com/paper/2412.14865/) (2024)
- [Torque-Aware Momentum](https://mlsanity.com/paper/2412.18790/) (2024)
- [Overcoming Non-stationary Dynamics with Evidential Proximal Policy Optimization](https://mlsanity.com/paper/2503.01468/) (2025)
- [Evaluating Temporal Plasticity in Foundation Time Series Models for Incremental Fine-tuning](https://mlsanity.com/paper/2504.14677/) (2025)
- [Plasticine: Accelerating Research in Plasticity-Motivated Deep Reinforcement Learning](https://mlsanity.com/paper/2504.17490/) (2025)
- [Preserving Plasticity in Continual Learning with Adaptive Linearity Injection](https://mlsanity.com/paper/2505.09486/) (2025)

## Abstract (arXiv)

In continual learning, plasticity refers to the ability of an agent to quickly adapt to new information. Neural networks are known to lose plasticity when processing non-stationary data streams. In this paper, we propose L2 Init, a simple approach for maintaining plasticity by incorporating in the loss function L2 regularization toward initial parameters. This is very similar to standard L2 regularization (L2), the only difference being that L2 regularizes toward the origin. L2 Init is simple to implement and requires selecting only a single hyper-parameter. The motivation for this method is the same as that of methods that reset neurons or parameter values. Intuitively, when recent losses are insensitive to particular parameters, these parameters should drift toward their initial values. This prepares parameters to adapt quickly to new tasks. On problems representative of different types of nonstationarity in continual supervised learning, we demonstrate that L2 Init most consistently mitigates plasticity loss compared to previously proposed approaches.