mlsanityData to 14 Oct 2026

Neuron-level Balance between Stability and Plasticity in Deep Reinforcement Learning

arXiv 2504.08000 ↗First posted 9 Apr 2025cs.AI cs.LGcited by 0

On the map: Deep reinforcement learning methods → Deep RL algorithms and representations

Abstract (arXiv)

In contrast to the human ability to continuously acquire knowledge, agents struggle with the stability-plasticity dilemma in deep reinforcement learning (DRL), which refers to the trade-off between retaining existing skills (stability) and learning new knowledge (plasticity). Current methods focus on balancing these two aspects at the network level, lacking sufficient differentiation and fine-grained control of individual neurons. To overcome this limitation, we propose Neuron-level Balance between Stability and Plasticity (NBSP) method, by taking inspiration from the observation that specific neurons are strongly relevant to task-relevant skills. Specifically, NBSP first (1) defines and identifies RL skill neurons that are crucial for knowledge retention through a goal-oriented method, and then (2) introduces a framework by employing gradient masking and experience replay techniques targeting these neurons to preserve the encoded existing skills while enabling adaptation to new tasks. Numerous experimental results on the Meta-World and Atari benchmarks demonstrate that NBSP significantly outperforms existing approaches in balancing stability and plasticity.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Deep RL algorithms and representations, one of 15 directions in Deep reinforcement learning methods. Builds on 37 of its 50 references that are in the map. First posted on arXiv 9 Apr 2025.

Builds on

37 of 50 references.

Closest ancestors · 1

Same direction, newest first

  1. 2024Reset & Distill: A Recipe for Overcoming Negative Transfer in Continual Reinforcement Learning8 Mar

Other references · 36

  1. 2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,885
  2. 2016Overcoming catastrophic forgetting in neural networkscited by 4,787
  3. 2013Playing Atari with Deep Reinforcement Learningcited by 3,274
  4. 2017Gradient Episodic Memory for Continual Learningcited by 1,898
  5. 2019Meta-World: A Benchmark and Evaluation for Multi-Task and Meta Reinforcement Learningcited by 1,204
  6. 2018Efficient Lifelong Learning with A-GEMcited by 1,058
  7. 2017PackNet: Adding Multiple Tasks to a Single Network by Iterative Pruningcited by 898
  8. 2023A Comprehensive Survey of Continual Learning: Theory, Method and Applicationcited by 872
  9. 2018Riemannian Walk for Incremental Learning: Understanding Forgetting and Intransigencecited by 742
  10. 2013An Empirical Investigation of Catastrophic Forgetting in Gradient-Based Neural Networkscited by 722
  11. 2020Deep Reinforcement Learning for Autonomous Driving: A Surveycited by 721
  12. 2018Learning Dexterous In-Hand Manipulationcited by 687
24 more
  1. 2021Knowledge Neurons in Pretrained Transformerscited by 601
  2. 2023Language Models Represent Space and Timecited by 316
  3. 2020Understanding the Role of Individual Units in a Deep Neural Networkcited by 255
  4. 2022The Primacy Bias in Deep Reinforcement Learningcited by 247
  5. 2023The Dormant Neuron Phenomenon in Deep Reinforcement Learningcited by 174
  6. 2020Multi-Task Reinforcement Learning with Soft Modularizationcited by 143
  7. 2024Language-Specific Neurons: The Key to Multilingual Capabilities in Large Language Modelscited by 126
  8. 2023Loss of Plasticity in Continual Deep Reinforcement Learningcited by 121
  9. 2018Identifying and Controlling Important Neurons in Neural Machine Translationcited by 111
  10. 2021Neuron-level Interpretation of Deep NLP Models: A Surveycited by 81
  11. 2023Deep Reinforcement Learning with Plasticity Injectioncited by 74
  12. 2022Finding Skill Neurons in Pre-trained Transformer-based Language Modelscited by 66
  13. 2023Maintaining Plasticity in Continual Learning via Regenerative Regularizationcited by 65
  14. 2024Towards Understanding Safety Alignment: A Mechanistic Perspective from Safety Neuronscited by 53
  15. 2018Pseudo-Rehearsal: Achieving Deep Reinforcement Learning without Catastrophic Forgettingcited by 46
  16. 2023Rosetta Neurons: Mining the Common Units in a Model Zoocited by 46
  17. 2023Achieving a Better Stability-Plasticity Trade-off via Auxiliary Networks in Continual Learningcited by 41
  18. 2022Modular Lifelong Reinforcement Learning via Neural Compositioncited by 40
  19. 2019Overcoming Catastrophic Forgetting by Neuron-level Plasticity Controlcited by 36
  20. 2022Disentangling Transfer in Continual Reinforcement Learningcited by 33
  21. 2023Revisiting Plasticity in Visual Reinforcement Learning: Data, Modules and Training Stagescited by 28
  22. 2023Prediction and Control in Continual Reinforcement Learningcited by 19
  23. 2023Continual Task Allocation in Meta-Policy Network via Sparse Promptingcited by 11
  24. 2024Let's Focus on Neuron: Neuron-Level Supervised Fine-tuning for Large Language Modelcited by 8

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026Balancing Plasticity and Stability with Fast and Slow Successor Features
  2. 2026Rethinking Plasticity in Deep Reinforcement Learning
  3. 2026Stagnant Neuron: Towards Understanding the Plasticity Loss in Multi-Agent Reinforcement Learning Value Factorization Methods
  4. 2025AltNet: Addressing the Plasticity-Stability Dilemma in Reinforcement Learning
  5. 2026The Rank and Gradient Lost in Non-stationarity: Sample Weight Decay for Mitigating Plasticity Loss in Reinforcement Learning
  6. 2026Continual Reinforcement Learning with Neuroevolution
  7. 2024Neuroplastic Expansion in Deep Reinforcement Learning
  8. 2025Mastering Continual Reinforcement Learning through Fine-Grained Sparse Network Allocation and Dormant Neuron Exploration