mlsanityData to 7 Oct 2026

SPHERE: Mitigating the Loss of Spectral Plasticity in Mixture-of-Experts for Deep Reinforcement Learning

arXiv 2605.04712 ↗First posted 6 May 2026cs.LGcited by 2

On the map: Mixture-of-experts → MoE in reinforcement learning and robotics

Abstract (arXiv)

In deep reinforcement learning (DRL), an agent is trained from a stream of experience. In a continual learning setting, such agents can suffer from plasticity loss: their ability to learn new skills from new experiences diminishes over training. Recently, Mixture-of-Experts (MoE) networks have been reported to enable scaling laws and facilitate the learning of diverse skills. However, in continual reinforcement learning settings, their performance can degenerate as learning proceeds, indicating a loss of plasticity. To address this, building on Neural Tangent Kernel (NTK) theory, we formalize the plasticity loss in MoE policies as a loss of spectral plasticity. We then derive a tractable proxy for spectral plasticity, one expressible in terms of individual expert feature matrices. Leveraging this proxy, we introduce SPHERE, a practical Parseval penalty tailored for MoE-based policies that alleviates the loss of spectral plasticity. On MetaWorld and HumanoidBench, SPHERE improves average success under continual RL by 133% and 50% over an unregularized MoE baseline, while maintaining higher spectral plasticity throughout training.

Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Placed in MoE in reinforcement learning and robotics, one of 12 directions in Mixture-of-experts. Builds on 34 of its 46 references that are in the map. First posted on arXiv 6 May 2026.

Builds on

34 of 46 references.

Closest ancestors · 4

Same direction, newest first

  1. 2025MoE-Loco: Mixture of Experts for Multitask Locomotion11 Mar
  2. 2024MENTOR: Mixture-of-Experts Network with Task-Oriented Perturbation for Visual Reinforcement Learning19 Oct
  3. 2024Mixture of Experts in a Mixture of RL settings26 Jun
  4. 2024Mixtures of Experts Unlock Parameter Scaling for Deep RL13 Feb

Other references · 30

  1. 2017Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layercited by 3,413
  2. 2018Spectral Normalization for Generative Adversarial Networkscited by 1,618
  3. 2019Meta-World: A Benchmark and Evaluation for Multi-Task and Meta Reinforcement Learningcited by 1,168
  4. 2024DeepSeekMoE: Towards Ultimate Expert Specialization in Mixture-of-Experts Language Modelscited by 834
  5. 2015Optimizing Neural Networks with Kronecker-factored Approximate Curvaturecited by 776
  6. 2019Wide Neural Networks of Any Depth Evolve as Linear Models Under Gradient Descentcited by 732
  7. 2022Mixture-of-Experts with Expert Choice Routingcited by 611
  8. 2022The Primacy Bias in Deep Reinforcement Learningcited by 242
  9. 2023The Dormant Neuron Phenomenon in Deep Reinforcement Learningcited by 171
  10. 2023Understanding plasticity in neural networkscited by 170
  11. 2021The Low-Rank Simplicity Bias in Deep Networkscited by 150
  12. 2017Practical Gauss-Newton Optimisation for Deep Learningcited by 139
18 more
  1. 2016A Kronecker-factored approximate Fisher matrix for convolution layerscited by 138
  2. 2020Implicit Under-Parameterization Inhibits Data-Efficient Deep Reinforcement Learningcited by 134
  3. 2022Understanding and Preventing Capacity Loss in Reinforcement Learningcited by 124
  4. 2023Loss of Plasticity in Continual Deep Reinforcement Learningcited by 118
  5. 2021Continual World: A Robotic Benchmark For Continual Reinforcement Learningcited by 112
  6. 2024HumanoidBench: Simulated Humanoid Benchmark for Whole-Body Locomotion and Manipulationcited by 101
  7. 2024Disentangling the Causes of Plasticity Loss in Neural Networkscited by 82
  8. 2020Transient Non-Stationarity and Generalisation in Deep Reinforcement Learningcited by 75
  9. 2024Learning Continually by Spectral Regularizationcited by 34
  10. 2023Revisiting Plasticity in Visual Reinforcement Learning: Data, Modules and Training Stagescited by 27
  11. 2023Directions of Curvature as an Explanation for Loss of Plasticitycited by 26
  12. 2024A Study of Plasticity Loss in On-Policy Deep Reinforcement Learningcited by 26
  13. 2025Mitigating Plasticity Loss in Continual Reinforcement Learning by Reducing Churncited by 22
  14. 2024Parseval Regularization for Continual Reinforcement Learningcited by 21
  15. 2024Don't flatten, tokenize! Unlocking the key to SoftMoE's efficacy in deep RLcited by 14
  16. 2025MoRE: Mixture of Residual Experts for Humanoid Lifelike Gaits Learning on Complex Terrainscited by 6
  17. 2024Theoretical characterisation of the Gauss-Newton conditioning in Neural Networkscited by 5
  18. 2025Spectral Collapse Drives Loss of Plasticity in Deep Continual Learningcited by 5

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 2.

2 papers.

  1. 2026RES-DARE: Failure-Aware Expert Adaptation and Rollback-Safe Self-Repair for Intrusion Detectioncited by 0
  2. 2026Beyond Forgetting: Diagnosing and Harnessing Shared Reasoning in Continual RLVRcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Dynamic Mixture of Experts Against Severe Distribution Shifts
  2. 2026Phase-Aware Mixture of Experts for Agentic Reinforcement Learning
  3. 2025Towards Stable and Effective Reinforcement Learning for Mixture-of-Experts
  4. 2021Probabilistic Mixture-of-Experts for Efficient Deep Reinforcement Learning
  5. 2026Temporally Extended Mixture-of-Experts Models
  6. 2020Contextual Policy Transfer in Reinforcement Learning Domains via Deep Mixtures-of-Experts
  7. 2025Plasticity-Aware Mixture of Experts for Learning Under QoE Shifts in Adaptive Video Streaming
  8. 2023Multi-Task Reinforcement Learning with Mixture of Orthogonal Experts