mlsanityData to 7 Oct 2026

Neuroplastic Expansion in Deep Reinforcement Learning

arXiv 2410.07994 ↗First posted 10 Oct 2024cs.LGcited by 17

Not on the map. Why?

Abstract (arXiv)

The loss of plasticity in learning agents, analogous to the solidification of neural pathways in biological brains, significantly impedes learning and adaptation in reinforcement learning due to its non-stationary nature. To address this fundamental challenge, we propose a novel approach, it Neuroplastic Expansion (NE), inspired by cortical expansion in cognitive science. NE maintains learnability and adaptability throughout the entire training process by dynamically growing the network from a smaller initial size to its full dimension. Our method is designed with three key components: (textit1) elastic topology generation based on potential gradients, (textit2) dormant neuron pruning to optimize network expressivity, and (textit3) neuron consolidation via experience review to strike a balance in the plasticity-stability dilemma. Extensive experiments demonstrate that NE effectively mitigates plasticity loss and outperforms state-of-the-art methods across various tasks in MuJoCo and DeepMind Control Suite environments. NE enables more adaptive learning in complex, dynamic environments, which represents a crucial step towards transitioning deep reinforcement learning from static, one-time training paradigms to more flexible, continually adapting models.

Key facts. Cited by 17 papers in the map (data through 7 Oct 2026). Builds on 49 of its 65 references that are in the map. First posted on arXiv 10 Oct 2024.

Builds on

49 of 65 references.

References · 49

  1. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  2. 2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,635
  3. 2015Continuous control with deep reinforcement learningcited by 3,279
  4. 2018Addressing Function Approximation Error in Actor-Critic Methodscited by 2,302
  5. 2016OpenAI Gymcited by 1,684
  6. 2018Experience Replay for Continual Learningcited by 1,074
  7. 2018Exploration by Random Network Distillationcited by 966
  8. 2018DeepMind Control Suitecited by 935
  9. 2019Rigging the Lottery: Making All Tickets Winnerscited by 491
  10. 2021Mastering Visual Continuous Control: Improved Data-Augmented Reinforcement Learningcited by 359
  11. 2022The Primacy Bias in Deep Reinforcement Learningcited by 242
  12. 2023The Dormant Neuron Phenomenon in Deep Reinforcement Learningcited by 171
37 more
  1. 2023Understanding plasticity in neural networkscited by 170
  2. 2023Bigger, Better, Faster: Human-level Atari with human-level efficiencycited by 145
  3. 2020Implicit Under-Parameterization Inhibits Data-Efficient Deep Reinforcement Learningcited by 134
  4. 2022Understanding and Preventing Capacity Loss in Reinforcement Learningcited by 124
  5. 2023Loss of Plasticity in Continual Deep Reinforcement Learningcited by 118
  6. 2022Learning Smooth Neural Functions via Lipschitz Regularizationcited by 86
  7. 2024Disentangling the Causes of Plasticity Loss in Neural Networkscited by 82
  8. 2024Mixtures of Experts Unlock Parameter Scaling for Deep RLcited by 75
  9. 2023For SALE: State-Action Representation Learning for Deep Reinforcement Learningcited by 73
  10. 2023Deep Reinforcement Learning with Plasticity Injectioncited by 71
  11. 2022The State of Sparse Training in Deep Reinforcement Learningcited by 60
  12. 2023Hyperparameters in Reinforcement Learning and How To Tune Themcited by 58
  13. 2024Normalization and effective learning rates in reinforcement learningcited by 55
  14. 2024Overestimation, Overfitting, and Plasticity in Actor-Critic: the Bitter Lesson of Reinforcement Learningcited by 50
  15. 2021Dynamic Sparse Training for Deep Reinforcement Learningcited by 49
  16. 2024Addressing Loss of Plasticity and Catastrophic Forgetting in Continual Learningcited by 49
  17. 2023DrM: Mastering Visual Reinforcement Learning through Dormant Ratio Minimizationcited by 45
  18. 2021A study on the plasticity of neural networkscited by 44
  19. 2023PLASTIC: Improving Input and Label Plasticity for Sample Efficient Reinforcement Learningcited by 42
  20. 2024Slow and Steady Wins the Race: Maintaining Plasticity with Hare and Tortoise Networkscited by 40
  21. 2024In value-based deep reinforcement learning, a pruned network is a good networkcited by 38
  22. 2023Small batch deep reinforcement learningcited by 35
  23. 2021Towards Better Plasticity-Stability Trade-off in Incremental Learning: A Simple Linear Connectorcited by 34
  24. 2023Language Model Alignment with Elastic Resetcited by 34
  25. 2020PoPS: Policy Pruning and Shrinking for Deep Reinforcement Learningcited by 30
  26. 2022RLx2: Training a Sparse Deep Reinforcement Learning Model from Scratchcited by 30
  27. 2024Weight Clipping for Deep Continual and Reinforcement Learningcited by 29
  28. 2023Revisiting Plasticity in Visual Reinforcement Learning: Data, Modules and Training Stagescited by 27
  29. 2023Directions of Curvature as an Explanation for Loss of Plasticitycited by 26
  30. 2024Mixture of Experts in a Mixture of RL settingscited by 21
  31. 2018Curiosity-driven Exploration for Mapless Navigation with Deep Reinforcement Learningcited by 14
  32. 2020The Effect of Multi-step Methods on Overestimation in Deep Reinforcement Learningcited by 14
  33. 2024Don't flatten, tokenize! Unlocking the key to SoftMoE's efficacy in deep RLcited by 14
  34. 2023JaxPruner: A concise library for sparsity researchcited by 13
  35. 2023Sample-Efficient and Safe Deep Reinforcement Learning via Reset Deep Ensemble Agentscited by 12
  36. 2019Natural-Logarithm-Rectified Activation Function in Convolutional Neural Networkscited by 8
  37. 2020Overcoming Catastrophic Forgetting in Graph Neural Networks with Experience Replaycited by 5

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 1; 2025: 10; 2026: 6.

17 papers.

Sort:

By area:

  1. 2024Plasticity Loss in Deep Reinforcement Learning: A Surveycited by 24
  2. 2025Stable Gradients for Stable Learning at Scale in Deep Reinforcement Learningcited by 22
  3. 2025XQC: Well-conditioned Optimization Accelerates Deep Reinforcement Learningcited by 17
  4. 2025Learning to Sample Effective and Diverse Prompts for Text-to-Image Generationcited by 16
  5. 2025Measure gradients, not activations! Enhancing neuronal activity in deep reinforcement learningcited by 14
  6. 2025Simplicial Embeddings Improve Sample Efficiency in Actor-Critic Agentscited by 12
  7. 2025Asymmetric Proximal Policy Optimization: mini-critics boost LLM reasoningcited by 9
  8. 2025Silent Neuron Theory and Plasticity Preservation for Deep Reinforcement Learning in Adaptive Video Streamingcited by 1
  9. 2025Growing with Experience: Growing Neural Networks in Deep Reinforcement Learningcited by 1
  10. 2025Rethinking the Role of Dynamic Sparse Training for Scalable Deep Reinforcement Learningcited by 1
  11. 2025A Simple Baseline for Stable and Plastic Neural Networkscited by 0
  12. 2026Rethinking Plasticity in Deep Reinforcement Learningcited by 0
5 more
  1. 2026Local Guidance, Global Impact: Gaussian-Reshaped Trust Region Unlocks Behavior Transitionscited by 0
  2. 2026Representation Learning Enables Scalable Multitask Deep Reinforcement Learningcited by 0
  3. 2026Stagnant Neuron: Towards Understanding the Plasticity Loss in Multi-Agent Reinforcement Learning Value Factorization Methodscited by 0
  4. 2026What Does Model Growth Really Add? Functional Capacity Beyond Parameter Countcited by 0
  5. 2026V-Simba: Unleashing the Architectural Potential of RL in Visual Continuous Controlcited by 0