mlsanityData to 7 Oct 2026

The Dormant Neuron Phenomenon in Deep Reinforcement Learning

arXiv 2302.12902 ↗First posted 24 Feb 2023cs.LGcited by 171

Not on the map. Why?

Abstract (arXiv)

In this work we identify the dormant neuron phenomenon in deep reinforcement learning, where an agent's network suffers from an increasing number of inactive neurons, thereby affecting network expressivity. We demonstrate the presence of this phenomenon across a variety of algorithms and environments, and highlight its effect on learning. To address this issue, we propose a simple and effective method (ReDo) that Recycles Dormant neurons throughout training. Our experiments demonstrate that ReDo maintains the expressive power of networks by reducing the number of dormant neurons and results in improved performance.

Key facts. Cited by 171 papers in the map (data through 7 Oct 2026). Builds on 45 of its 72 references that are in the map. First posted on arXiv 24 Feb 2023.

Builds on

45 of 72 references.

References · 45

  1. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  2. 2020Scaling Laws for Neural Language Modelscited by 6,411
  3. 2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,635
  4. 2015Continuous control with deep reinforcement learningcited by 3,279
  5. 2021Scaling Vision Transformerscited by 993
  6. 2018IMPALA: Scalable Distributed Deep-RL with Importance Weighted Actor-Learner Architecturescited by 866
  7. 2021Deep Reinforcement Learning at the Edge of the Statistical Precipicecited by 781
  8. 2017Rainbow: Combining Improvements in Deep Reinforcement Learningcited by 774
  9. 2019When to Trust Your Model: Model-Based Policy Optimizationcited by 753
  10. 2017Lifelong Learning with Dynamically Expandable Networkscited by 689
  11. 2020Image Augmentation Is All You Need: Regularizing Deep Reinforcement Learning from Pixelscited by 573
  12. 2017Deep Learning Scaling is Predictable, Empiricallycited by 556
33 more
  1. 2019Model-Based Reinforcement Learning for Ataricited by 495
  2. 2021Randomized Ensembled Double Q-Learning: Learning Fast Without a Modelcited by 278
  3. 2022The Primacy Bias in Deep Reinforcement Learningcited by 242
  4. 2020Revisiting Fundamentals of Experience Replaycited by 171
  5. 2018Deep Reinforcement Learning and the Deadly Triadcited by 170
  6. 2020Implicit Under-Parameterization Inhibits Data-Efficient Deep Reinforcement Learningcited by 134
  7. 2022Understanding and Preventing Capacity Loss in Reinforcement Learningcited by 124
  8. 2018Dopamine: A Research Framework for Deep Reinforcement Learningcited by 121
  9. 2019When to use parametric models in reinforcement learning?cited by 118
  10. 2021Dropout Q-Functions for Doubly Efficient Reinforcement Learningcited by 108
  11. 2021Stabilizing Deep Q-Learning with ConvNets and Vision Transformers under Data Augmentationcited by 101
  12. 2020Improving Generalization in Reinforcement Learning with Mixture Regularizationcited by 81
  13. 2021Continual Backprop: Stochastic Gradient Descent with Persistent Randomnesscited by 79
  14. 2020Transient Non-Stationarity and Generalisation in Deep Reinforcement Learningcited by 75
  15. 2019Diagnosing Bottlenecks in Deep Q-learning Algorithmscited by 73
  16. 2021DR3: Value-Based Deep Reinforcement Learning Requires Explicit Regularizationcited by 66
  17. 2022GradMax: Growing Neural Networks using Gradient Informationcited by 64
  18. 2017NeST: A Neural Network Synthesis Tool Based on a Grow-and-Prune Paradigmcited by 60
  19. 2022The State of Sparse Training in Deep Reinforcement Learningcited by 60
  20. 2019Splitting Steepest Descent for Growing Neural Architecturescited by 52
  21. 2021Dynamic Sparse Training for Deep Reinforcement Learningcited by 49
  22. 2019The Impact of Neural Network Overparameterization on Gradient Confusion and Stochastic Gradient Descentcited by 46
  23. 2021A study on the plasticity of neural networkscited by 44
  24. 2021SECANT: Self-Expert Cloning for Zero-Shot Generalization of Visual Policiescited by 42
  25. 2020Interference and Generalization in Temporal Difference Learningcited by 41
  26. 2022Fortuitous Forgetting in Connectionist Networkscited by 39
  27. 2022RLx2: Training a Sparse Deep Reinforcement Learning Model from Scratchcited by 30
  28. 2020RIFLE: Backpropagation in Depth for Deep Transfer Learning through Re-Initializing the Fully-connected LayErcited by 27
  29. 2022An Empirical Study of Implicit Regularization in Deep Offline RLcited by 24
  30. 2021Single-Shot Pruning for Offline Reinforcement Learningcited by 21
  31. 2021The Impact of Reinitialization on Generalization in Convolutional Neural Networkscited by 20
  32. 2021Knowledge Evolution in Neural Networkscited by 18
  33. 2022When Does Re-initialization Work?cited by 12

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 13; 2024: 53; 2025: 48; 2026: 57.

171 papers.

Sort:

By area:

  1. 2024One Step Diffusion via Shortcut Modelscited by 373
  2. 2023Bigger, Better, Faster: Human-level Atari with human-level efficiencycited by 145
  3. 2024Stop Regressing: Training Value Functions via Classification for Scalable Deep RLcited by 136
  4. 2024Disentangling the Causes of Plasticity Loss in Neural Networkscited by 82
  5. 2024Mixtures of Experts Unlock Parameter Scaling for Deep RLcited by 75
  6. 2023Deep Reinforcement Learning with Plasticity Injectioncited by 71
  7. 2024SimBa: Simplicity Bias for Scaling Up Parameters in Deep Reinforcement Learningcited by 70
  8. 2023Maintaining Plasticity in Continual Learning via Regenerative Regularizationcited by 63
  9. 2024Normalization and effective learning rates in reinforcement learningcited by 55
  10. 2024Overestimation, Overfitting, and Plasticity in Actor-Critic: the Bitter Lesson of Reinforcement Learningcited by 50
  11. 2023DrM: Mastering Visual Reinforcement Learning through Dormant Ratio Minimizationcited by 45
  12. 2023PLASTIC: Improving Input and Label Plasticity for Sample Efficient Reinforcement Learningcited by 42
38 more
  1. 2023Maintaining Plasticity in Deep Continual Learningcited by 42
  2. 2024Diagnosing and Re-learning for Balanced Multimodal Learningcited by 40
  3. 2024Kinetix: Investigating the Training of General Agents through Open-Ended Physics-Based Control Taskscited by 40
  4. 2025Towards General-Purpose Model-Free Reinforcement Learningcited by 40
  5. 2024In value-based deep reinforcement learning, a pruned network is a good networkcited by 38
  6. 2023Small batch deep reinforcement learningcited by 35
  7. 2025Hyperspherical Normalization for Scalable Deep Reinforcement Learningcited by 35
  8. 2024Learning Continually by Spectral Regularizationcited by 34
  9. 2024Confronting Reward Overoptimization for Diffusion Models: A Perspective of Inductive and Primacy Biasescited by 29
  10. 2024Weight Clipping for Deep Continual and Reinforcement Learningcited by 29
  11. 2023Revisiting Plasticity in Visual Reinforcement Learning: Data, Modules and Training Stagescited by 27
  12. 2024No Representation, No Trust: Connecting Representation, Collapse, and Trust Issues in PPOcited by 27
  13. 2023Directions of Curvature as an Explanation for Loss of Plasticitycited by 26
  14. 2024A Study of Plasticity Loss in On-Policy Deep Reinforcement Learningcited by 26
  15. 2024Plasticity Loss in Deep Reinforcement Learning: A Surveycited by 24
  16. 2025Mitigating Plasticity Loss in Continual Reinforcement Learning by Reducing Churncited by 22
  17. 2025Stable Gradients for Stable Learning at Scale in Deep Reinforcement Learningcited by 22
  18. 2024On the consistency of hyper-parameter selection in value-based deep reinforcement learningcited by 21
  19. 2024Mixture of Experts in a Mixture of RL settingscited by 21
  20. 2024Parseval Regularization for Continual Reinforcement Learningcited by 21
  21. 2024MENTOR: Mixture-of-Experts Network with Task-Oriented Perturbation for Visual Reinforcement Learningcited by 20
  22. 2024Dissecting Deep RL with High Update Ratios: Combatting Value Divergencecited by 17
  23. 2024Neuroplastic Expansion in Deep Reinforcement Learningcited by 17
  24. 2024Non-Stationary Learning of Neural Networks with Automatic Soft Parameter Resetcited by 17
  25. 2025Optimizers Qualitatively Alter Solutions And We Should Leverage Thiscited by 17
  26. 2024ACE : Off-Policy Actor-Critic with Causality-Aware Entropy Regularizationcited by 16
  27. 2024Fast TRAC: A Parameter-Free Optimizer for Lifelong Reinforcement Learningcited by 16
  28. 2024Improving Deep Reinforcement Learning by Reducing the Chain Effect of Value and Policy Churncited by 16
  29. 2024MAD-TD: Model-Augmented Data stabilizes High Update Ratio RLcited by 16
  30. 2024Deep Policy Gradient Methods Without Batch Updates, Target Networks, or Replay Bufferscited by 16
  31. 2025Learning to Sample Effective and Diverse Prompts for Text-to-Image Generationcited by 16
  32. 2024Can Learned Optimization Make Reinforcement Learning Less Difficult?cited by 14
  33. 2024Don't flatten, tokenize! Unlocking the key to SoftMoE's efficacy in deep RLcited by 14
  34. 2024Plastic Learning with Deep Fourier Featurescited by 14
  35. 2025Value-Based Deep RL Scales Predictablycited by 14
  36. 2025Measure gradients, not activations! Enhancing neuronal activity in deep reinforcement learningcited by 14
  37. 2023JaxPruner: A concise library for sparsity researchcited by 13
  38. 2024Scaling Offline Model-Based RL via Jointly-Optimized World-Action Model Pretrainingcited by 13

50 most cited of 171 All 171 (JSON)