mlsanityData to 14 Oct 2026

Stagnant Neuron: Towards Understanding the Plasticity Loss in Multi-Agent Reinforcement Learning Value Factorization Methods

arXiv 2606.25335 ↗First posted 24 Jun 2026cs.LGcited by 0

Not on the map. Why?

Abstract (arXiv)

Multi-Agent Reinforcement Learning (MARL) value factorization methods can suffer from a loss of plasticity, gradually failing to adapt when transferring to new task instances. We trace this issue to stagnant neurons, units whose gradient updates become negligibly small relative to their weights, thereby hindering learning. While existing plasticity injection methods exist, they prove ineffective for such neurons. To address this, we propose Knowledge-retentive Neuron-level PlastIcity Focusing InjEction (KNIFE), a novel method that directly targets stagnant neurons. KNIFE replaces each stagnant neuron with a composite unit comprising three specialized components: a frozen knowledge neuron to preserve acquired knowledge, a re-initialized active neuron to restore learning capacity, and a compensation neuron to ensure the combined output matches the original, thus maintaining previous learned cooperation knowledge. Extensive experiments on SMACv2, predator-prey, and matrix games demonstrate that KNIFE significantly outperforms state-of-the-art plasticity injection methods.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 30 of its 39 references that are in the map. First posted on arXiv 24 Jun 2026.

Builds on

30 of 39 references.

References · 30

  1. 2016Overcoming catastrophic forgetting in neural networkscited by 4,787
  2. 2017Multi-Agent Actor-Critic for Mixed Cooperative-Competitive Environmentscited by 1,558
  3. 2017Memory Aware Synapses: Learning what (not) to forgetcited by 1,105
  4. 2016HyperNetworkscited by 1,027
  5. 2021The Surprising Effectiveness of PPO in Cooperative, Multi-Agent Gamescited by 942
  6. 2017PackNet: Adding Multiple Tasks to a Single Network by Iterative Pruningcited by 898
  7. 2018QMIX: Monotonic Value Function Factorisation for Deep Multi-Agent Reinforcement Learningcited by 708
  8. 2019The StarCraft Multi-Agent Challengecited by 597
  9. 2019QTRAN: Learning to Factorize with Transformation for Cooperative Multi-Agent Reinforcement Learningcited by 379
  10. 2020QPLEX: Duplex Dueling Multi-Agent Q-Learningcited by 270
  11. 2022The Primacy Bias in Deep Reinforcement Learningcited by 247
  12. 2020Weighted QMIX: Expanding Monotonic Value Function Factorisation for Deep Multi-Agent Reinforcement Learningcited by 197
18 more
  1. 2023The Dormant Neuron Phenomenon in Deep Reinforcement Learningcited by 174
  2. 2023Understanding plasticity in neural networkscited by 174
  3. 2018Graph Convolutional Reinforcement Learningcited by 152
  4. 2022SMACv2: An Improved Benchmark for Cooperative Multi-Agent Reinforcement Learningcited by 122
  5. 2023A Definition of Continual Reinforcement Learningcited by 101
  6. 2020Qatten: A General Framework for Cooperative Multiagent Reinforcement Learningcited by 81
  7. 2021Continual Backprop: Stochastic Gradient Descent with Persistent Randomnesscited by 80
  8. 2023Deep Reinforcement Learning with Plasticity Injectioncited by 74
  9. 2018Pseudo-Rehearsal: Achieving Deep Reinforcement Learning without Catastrophic Forgettingcited by 46
  10. 2021UPDeT: Universal Multi-agent Reinforcement Learning via Policy Decoupling with Transformerscited by 39
  11. 2024Learning Continually by Spectral Regularizationcited by 34
  12. 2021DFAC Framework: Factorizing the Value Function via Quantile Mixture for Multi-Agent Distributional Q-Learningcited by 26
  13. 2023RiskQ: Risk-sensitive Multi-Agent Reinforcement Learning Value Factorizationcited by 20
  14. 2024Neuroplastic Expansion in Deep Reinforcement Learningcited by 18
  15. 2025Measure gradients, not activations! Enhancing neuronal activity in deep reinforcement learningcited by 14
  16. 2025MEAL: A Benchmark for Continual Multi-Agent Reinforcement Learningcited by 11
  17. 2025Experience Replay Addresses Loss of Plasticity in Continual Learningcited by 4
  18. 2026Retaining Suboptimal Actions to Follow Shifting Optima in Multi-Agent Reinforcement Learningcited by 3

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025Mastering Continual Reinforcement Learning through Fine-Grained Sparse Network Allocation and Dormant Neuron Exploration
  2. 2025Sparsity-Driven Plasticity in Multi-Task Reinforcement Learning
  3. 2025Neuron-level Balance between Stability and Plasticity in Deep Reinforcement Learning
  4. 2026Rethinking Plasticity in Deep Reinforcement Learning
  5. 2025Plasticine: Accelerating Research in Plasticity-Motivated Deep Reinforcement Learning
  6. 2025AltNet: Addressing the Plasticity-Stability Dilemma in Reinforcement Learning
  7. 2024Plasticity Loss in Deep Reinforcement Learning: A Survey
  8. 2026The Rank and Gradient Lost in Non-stationarity: Sample Weight Decay for Mitigating Plasticity Loss in Reinforcement Learning