mlsanityData to 7 Oct 2026

Plasticity Loss in Deep Reinforcement Learning: A Survey

arXiv 2411.04832 ↗First posted 7 Nov 2024cs.AI cs.LGcited by 24

Not on the map. Why?

Abstract (arXiv)

Plasticity refers to a network's ability to adapt to changing data distributions, which is crucial for the successful training of deep reinforcement learning agents. Loss of plasticity causes performance plateaus and contributes to scaling failures, overestimation bias, and insufficient exploration. To deepen the understanding of plasticity loss, we propose a unified definition, examine its drivers and pathologies, and organize over 50 mitigation strategies into the first comprehensive taxonomy of the field. Our analysis shows gaps in current evaluation practices and reveals that general regularization techniques often outperform domain-specific interventions. Future research should prioritize understanding the mechanisms underlying plasticity loss.

Key facts. Cited by 24 papers in the map (data through 7 Oct 2026). Builds on 70 of its 105 references that are in the map. First posted on arXiv 7 Nov 2024.

Builds on

70 of 105 references.

References · 70

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  3. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,160
  4. 2015Distilling the Knowledge in a Neural Networkcited by 10,076
  5. 2015Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shiftcited by 6,979
  6. 2016Layer Normalizationcited by 5,579
  7. 2017Neural Discrete Representation Learningcited by 4,650
  8. 2018Addressing Function Approximation Error in Actor-Critic Methodscited by 2,302
  9. 2018Spectral Normalization for Generative Adversarial Networkscited by 1,618
  10. 2020Sharpness-Aware Minimization for Efficiently Improving Generalizationcited by 1,404
  11. 2019Mastering Atari, Go, Chess and Shogi by Planning with a Learned Modelcited by 1,327
  12. 2019Dota 2 with Large Scale Deep Reinforcement Learningcited by 1,084
38 more
  1. 2015Fast and Accurate Deep Network Learning by Exponential Linear Units (ELUs)cited by 1,061
  2. 2018DeepMind Control Suitecited by 935
  3. 2018IMPALA: Scalable Distributed Deep-RL with Importance Weighted Actor-Learner Architecturescited by 866
  4. 2017A Distributional Perspective on Reinforcement Learningcited by 743
  5. 2013An Empirical Investigation of Catastrophic Forgetting in Gradient-Based Neural Networkscited by 711
  6. 2019Solving Rubik's Cube with a Robot Handcited by 699
  7. 2020Image Augmentation Is All You Need: Regularizing Deep Reinforcement Learning from Pixelscited by 573
  8. 2019Model-Based Reinforcement Learning for Ataricited by 495
  9. 2019Leveraging Procedural Generation to Benchmark Reinforcement Learningcited by 478
  10. 2020Reinforcement Learning with Augmented Datacited by 452
  11. 2021Mastering Visual Continuous Control: Improved Data-Augmented Reinforcement Learningcited by 359
  12. 2017Parseval Networks: Improving Robustness to Adversarial Examplescited by 288
  13. 2021Randomized Ensembled Double Q-Learning: Learning Fast Without a Modelcited by 278
  14. 2017Count-Based Exploration with Neural Density Modelscited by 255
  15. 2019Understanding and Improving Layer Normalizationcited by 213
  16. 2020Implementation Matters in Deep Policy Gradients: A Case Study on PPO and TRPOcited by 184
  17. 2023The Dormant Neuron Phenomenon in Deep Reinforcement Learningcited by 171
  18. 2018Deep Reinforcement Learning and the Deadly Triadcited by 170
  19. 2021The Low-Rank Simplicity Bias in Deep Networkscited by 150
  20. 2023Bigger, Better, Faster: Human-level Atari with human-level efficiencycited by 145
  21. 2024Stop Regressing: Training Value Functions via Classification for Scalable Deep RLcited by 136
  22. 2022Understanding and Preventing Capacity Loss in Reinforcement Learningcited by 124
  23. 2023Loss of Plasticity in Continual Deep Reinforcement Learningcited by 118
  24. 2024Bigger, Regularized, Optimistic: scaling for compute and sample-efficient continuous controlcited by 104
  25. 2023A Definition of Continual Reinforcement Learningcited by 98
  26. 2019CrossQ: Batch Normalization in Deep Reinforcement Learning for Greater Sample Efficiency and Simplicitycited by 89
  27. 2016Understanding and Improving Convolutional Neural Networks via Concatenated Rectified Linear Unitscited by 86
  28. 2024Disentangling the Causes of Plasticity Loss in Neural Networkscited by 82
  29. 2020Transient Non-Stationarity and Generalisation in Deep Reinforcement Learningcited by 75
  30. 2024Mixtures of Experts Unlock Parameter Scaling for Deep RLcited by 75
  31. 2023Deep Reinforcement Learning with Plasticity Injectioncited by 71
  32. 2024Simplifying Deep Temporal Difference Learningcited by 71
  33. 2018Improving Regression Performance with Distributional Lossescited by 66
  34. 2021DR3: Value-Based Deep Reinforcement Learning Requires Explicit Regularizationcited by 66
  35. 2021Spectral Normalisation for Deep Reinforcement Learning: an Optimisation Perspectivecited by 64
  36. 2021On The Effect of Auxiliary Tasks on Representation Dynamicscited by 56
  37. 2024Normalization and effective learning rates in reinforcement learningcited by 55
  38. 2021The Difficulty of Passive Learning in Deep Reinforcement Learningcited by 50

50 most cited of 70 All 70 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 13; 2026: 11.

24 papers.

Sort:

By area:

  1. 2025Hyperspherical Normalization for Scalable Deep Reinforcement Learningcited by 35
  2. 2025The Reasoning Boundary Paradox: How Reinforcement Learning Constrains Language Modelscited by 17
  3. 2025A Survey of Continual Reinforcement Learningcited by 16
  4. 2025Network Sparsity Unlocks the Scaling Potential of Deep Reinforcement Learningcited by 11
  5. 2025Activation Function Design Sustains Plasticity in Continual Learningcited by 6
  6. 2025UNIDOOR: A Universal Framework for Action-Level Backdoor Attacks in Deep Reinforcement Learningcited by 5
  7. 2025Stabilizing Policy Gradients for Sample-Efficient Reinforcement Learning in LLM Reasoningcited by 5
  8. 2026Weight Decay Improves Language Model Plasticitycited by 4
  9. 2025Mastering Continual Reinforcement Learning through Fine-Grained Sparse Network Allocation and Dormant Neuron Explorationcited by 2
  10. 2025Plasticine: Accelerating Research in Plasticity-Motivated Deep Reinforcement Learningcited by 2
  11. 2025Sparsity-Driven Plasticity in Multi-Task Reinforcement Learningcited by 2
  12. 2025Reusing Overtrained Language Models Saturates Scalingcited by 2
12 more
  1. 2025Rethinking the Role of Dynamic Sparse Training for Scalable Deep Reinforcement Learningcited by 1
  2. 2025Understanding and Improving Hyperbolic Deep Reinforcement Learningcited by 1
  3. 2026Beyond Single-Model Optimization: Preserving Plasticity in Continual Reinforcement Learningcited by 1
  4. 2026Angel or Demon: Investigating the Plasticity Interventions' Impact on Backdoor Threats in Deep Reinforcement Learningcited by 1
  5. 2026Safe Continual Reinforcement Learning in Non-stationary Environmentscited by 0
  6. 2026On the Stability of Growth in Structural Plasticity: Forward-Active yet Backward-Starvedcited by 0
  7. 2026Learning in Low-Dimensional Subspaces: Orthogonal Bottlenecks for Reinforcement Learningcited by 0
  8. 2026Beyond One-Size-Fits-All: Diagnosis-Driven Online Reinforcement Learning with Offline Priorscited by 0
  9. 2026Rank Collapse Is Recoverable, Growing |Q| Is Not: Out-of-Sample Early Warning for Value Divergence in High-UTD Soft Actor-Criticcited by 0
  10. 2026Policy Plasticity Matters in Offline-to-Online Reinforcement Learning: Refitting Offline Policies for Online Adaptationcited by 0
  11. 2026From Static Policies to Adaptive Priors in Offline Reinforcement Learningcited by 0
  12. 2026Continual Reinforcement Learning with Neuroevolutioncited by 0