mlsanityData to 14 Oct 2026

Understanding plasticity in neural networks

arXiv 2303.01486 ↗First posted 2 Mar 2023cs.LGcited by 174

Not on the map. Why?

Cited by 73 in Deep reinforcement learning methods, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Plasticity, the ability of a neural network to quickly change its predictions in response to new information, is essential for the adaptability and robustness of deep reinforcement learning systems. Deep neural networks are known to lose plasticity over the course of training even in relatively simple learning problems, but the mechanisms driving this phenomenon are still poorly understood. This paper conducts a systematic empirical analysis into plasticity loss, with the goal of understanding the phenomenon mechanistically in order to guide the future development of targeted solutions. We find that loss of plasticity is deeply connected to changes in the curvature of the loss landscape, but that it often occurs in the absence of saturated units. Based on this insight, we identify a number of parameterization and optimization design choices which enable networks to better preserve plasticity over the course of training. We validate the utility of these findings on larger-scale RL benchmarks in the Arcade Learning Environment.

Key facts. Cited by 174 papers in the map (data through 14 Oct 2026). Builds on 47 of its 63 references that are in the map. First posted on arXiv 2 Mar 2023.

Builds on

47 of 63 references.

References · 47

  1. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  2. 2014Adam: A Method for Stochastic Optimizationcited by 37,240
  3. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
  4. 2020Scaling Laws for Neural Language Modelscited by 6,586
  5. 2016Layer Normalizationcited by 5,669
  6. 2016Overcoming catastrophic forgetting in neural networkscited by 4,787
  7. 2015Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classificationcited by 3,611
  8. 2015Deep Reinforcement Learning with Double Q-learningcited by 1,745
  9. 2017Visualizing the Loss Landscape of Neural Netscited by 1,130
  10. 2018Experience Replay for Continual Learningcited by 1,095
  11. 2015Dueling Network Architectures for Deep Reinforcement Learningcited by 785
  12. 2019Linear Mode Connectivity and the Lottery Ticket Hypothesiscited by 637
35 more
  1. 2017Sharp Minima Can Generalize For Deep Netscited by 466
  2. 2021Gradient Descent on Neural Networks Typically Occurs at the Edge of Stabilitycited by 427
  3. 2017Measuring Catastrophic Forgetting in Neural Networkscited by 360
  4. 2018How Does Batch Normalization Help Optimization?cited by 357
  5. 2022How Do Vision Transformers Work?cited by 307
  6. 2019An Investigation into Neural Net Optimization via Hessian Eigenvalue Densitycited by 292
  7. 2016Exponential expressivity in deep neural networks through transient chaoscited by 261
  8. 2022The Primacy Bias in Deep Reinforcement Learningcited by 247
  9. 2017Overcoming Catastrophic Forgetting by Incremental Moment Matchingcited by 243
  10. 2020The large learning rate phase of deep learning: the catapult mechanismcited by 216
  11. 2016Deep Information Propagationcited by 195
  12. 2019Learning to Remember: A Synaptic Plasticity Driven Framework for Continual Learningcited by 182
  13. 2017The Shattered Gradients Problem: If resnets are the answer, then what is the question?cited by 157
  14. 2021A Universal Law of Robustness via Isoperimetrycited by 151
  15. 2020The Break-Even Point on Optimization Trajectories of Deep Neural Networkscited by 150
  16. 2020Implicit Under-Parameterization Inhibits Data-Efficient Deep Reinforcement Learningcited by 134
  17. 2022Understanding and Preventing Capacity Loss in Reinforcement Learningcited by 125
  18. 2018Invariance, Causality and Robustnesscited by 123
  19. 2019Are All Layers Created Equal?cited by 94
  20. 2017Mean Field Residual Networks: On the Edge of Chaoscited by 83
  21. 2018Kickstarting Deep Reinforcement Learningcited by 81
  22. 2021Continual Backprop: Stochastic Gradient Descent with Persistent Randomnesscited by 80
  23. 2020In Search of Robust Measures of Generalizationcited by 79
  24. 2020Transient Non-Stationarity and Generalisation in Deep Reinforcement Learningcited by 75
  25. 2021Spectral Normalisation for Deep Reinforcement Learning: an Optimisation Perspectivecited by 65
  26. 2019A Mean Field Theory of Batch Normalizationcited by 58
  27. 2021Rapid training of deep neural networks without skip connections or normalization layers using Deep Kernel Shapingcited by 52
  28. 2019Stiffness: A New Perspective on Generalization in Neural Networkscited by 46
  29. 2021A study on the plasticity of neural networkscited by 44
  30. 2021A Loss Curvature Perspective on Training Instability in Deep Learningcited by 40
  31. 2022Deep Learning without Shortcuts: Shaping the Kernel with Tailored Rectifierscited by 28
  32. 2022An Empirical Study of Implicit Regularization in Deep Offline RLcited by 24
  33. 2021Adaptive Rational Activations to Boost Deep Reinforcement Learningcited by 23
  34. 2020On Catastrophic Interference in Atari 2600 Gamescited by 13
  35. 2020Enabling Continual Learning with Differentiable Hebbian Plasticitycited by 8

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 18; 2024: 45; 2025: 49; 2026: 62.

174 papers.

Sort:

By area:

  1. 2023A Definition of Continual Reinforcement Learningcited by 101
  2. 2024Disentangling the Causes of Plasticity Loss in Neural Networkscited by 86
  3. 2024Simplifying Deep Temporal Difference Learningcited by 78
  4. 2024SimBa: Simplicity Bias for Scaling Up Parameters in Deep Reinforcement Learningcited by 76
  5. 2023Deep Reinforcement Learning with Plasticity Injectioncited by 74
  6. 2023Maintaining Plasticity in Continual Learning via Regenerative Regularizationcited by 65
  7. 2024Normalization and effective learning rates in reinforcement learningcited by 58
  8. 2024Overestimation, Overfitting, and Plasticity in Actor-Critic: the Bitter Lesson of Reinforcement Learningcited by 50
  9. 2024Addressing Loss of Plasticity and Catastrophic Forgetting in Continual Learningcited by 50
  10. 2023DrM: Mastering Visual Reinforcement Learning through Dormant Ratio Minimizationcited by 47
  11. 2023Maintaining Plasticity in Deep Continual Learningcited by 43
  12. 2023Improving Language Plasticity via Pretraining with Active Forgettingcited by 43
38 more
  1. 2023PLASTIC: Improving Input and Label Plasticity for Sample Efficient Reinforcement Learningcited by 42
  2. 2024Learning with 3D rotations, a hitchhiker's guide to SO(3)cited by 42
  3. 2024In value-based deep reinforcement learning, a pruned network is a good networkcited by 39
  4. 2025Hyperspherical Normalization for Scalable Deep Reinforcement Learningcited by 38
  5. 2023Continual Learning as Computationally Constrained Reinforcement Learningcited by 37
  6. 2023Small batch deep reinforcement learningcited by 36
  7. 2025CaRL: Learning Scalable Planning Policies with Simple Rewardscited by 35
  8. 2024Learning Continually by Spectral Regularizationcited by 34
  9. 2024Confronting Reward Overoptimization for Diffusion Models: A Perspective of Inductive and Primacy Biasescited by 30
  10. 2024Weight Clipping for Deep Continual and Reinforcement Learningcited by 29
  11. 2023Revisiting Plasticity in Visual Reinforcement Learning: Data, Modules and Training Stagescited by 28
  12. 2024No Representation, No Trust: Connecting Representation, Collapse, and Trust Issues in PPOcited by 27
  13. 2023Directions of Curvature as an Explanation for Loss of Plasticitycited by 26
  14. 2024A Study of Plasticity Loss in On-Policy Deep Reinforcement Learningcited by 26
  15. 2023Symmetry Induces Structure and Constraint of Learningcited by 25
  16. 2024Streaming Deep Reinforcement Learning Finally Workscited by 25
  17. 2025Stable Gradients for Stable Learning at Scale in Deep Reinforcement Learningcited by 23
  18. 2024Mixture of Experts in a Mixture of RL settingscited by 21
  19. 2024Parseval Regularization for Continual Reinforcement Learningcited by 21
  20. 2024Neuroplastic Expansion in Deep Reinforcement Learningcited by 18
  21. 2024Deep Policy Gradient Methods Without Batch Updates, Target Networks, or Replay Bufferscited by 18
  22. 2024Dissecting Deep RL with High Update Ratios: Combatting Value Divergencecited by 17
  23. 2024Fast TRAC: A Parameter-Free Optimizer for Lifelong Reinforcement Learningcited by 17
  24. 2024MAD-TD: Model-Augmented Data stabilizes High Update Ratio RLcited by 16
  25. 2025Value-Based Deep RL Scales Predictablycited by 15
  26. 2024Can Learned Optimization Make Reinforcement Learning Less Difficult?cited by 14
  27. 2024Don't flatten, tokenize! Unlocking the key to SoftMoE's efficacy in deep RLcited by 14
  28. 2024Plastic Learning with Deep Fourier Featurescited by 14
  29. 2025Measure gradients, not activations! Enhancing neuronal activity in deep reinforcement learningcited by 14
  30. 2023Overcoming the Stability Gap in Continual Learningcited by 13
  31. 2025The Impact of On-Policy Parallelized Data Collection on Deep Reinforcement Learning Networkscited by 13
  32. 2024Discounted Adaptive Online Learning: Towards Better Regularizationcited by 12
  33. 2025Plasticity as the Mirror of Empowermentcited by 12
  34. 2025Simplicial Embeddings Improve Sample Efficiency in Actor-Critic Agentscited by 12
  35. 2024Position: Lifetime tuning is incompatible with continual reinforcement learningcited by 11
  36. 2024Self-Normalized Resets for Plasticity in Continual Learningcited by 11
  37. 2024Adam on Local Time: Addressing Nonstationarity in RL with Relative Adam Timestepscited by 11
  38. 2025Network Sparsity Unlocks the Scaling Potential of Deep Reinforcement Learningcited by 11

50 most cited of 174 All 174 (JSON)