mlsanityData to 7 Oct 2026

Dynamic Mixture of Experts Against Severe Distribution Shifts

arXiv 2511.18987 ↗First posted 24 Nov 2025cs.LG cs.AIcited by 0

On the map: Mixture-of-experts → Mixture of LoRA experts

Abstract (arXiv)

The challenge of building neural networks that can continuously learn and adapt to evolving data streams is central to the fields of continual learning (CL) and reinforcement learning (RL). This lifelong learning problem is often framed in terms of the plasticity-stability dilemma, focusing on issues like loss of plasticity and catastrophic forgetting. Unlike neural networks, biological brains maintain plasticity through capacity growth, inspiring researchers to explore similar approaches in artificial networks, such as adding capacity dynamically. Prior solutions often lack parameter efficiency or depend on explicit task indices, but Mixture-of-Experts (MoE) architectures offer a promising alternative by specializing experts for distinct distributions. This paper aims to evaluate a DynamicMoE approach for continual and reinforcement learning environments and benchmark its effectiveness against existing network expansion methods.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Mixture of LoRA experts, one of 12 directions in Mixture-of-experts. Builds on 11 of its 13 references that are in the map. First posted on arXiv 24 Nov 2025.

Builds on

11 of 13 references.

References · 11

  1. 2017Proximal Policy Optimization Algorithmscited by 14,441
  2. 2016Progressive Neural Networkscited by 1,332
  3. 2015Net2Net: Accelerating Learning via Knowledge Transfercited by 245
  4. 2024Craftax: A Lightning-Fast Benchmark for Open-Ended Reinforcement Learningcited by 80
  5. 2023Deep Reinforcement Learning with Plasticity Injectioncited by 71
  6. 2024SimBa: Simplicity Bias for Scaling Up Parameters in Deep Reinforcement Learningcited by 70
  7. 2023Continual Learning as Computationally Constrained Reinforcement Learningcited by 35
  8. 2024Mixture of Experts in a Mixture of RL settingscited by 21
  9. 2024MENTOR: Mixture-of-Experts Network with Task-Oriented Perturbation for Visual Reinforcement Learningcited by 20
  10. 2023Composable Function-preserving Expansions for Transformer Architecturescited by 11
  11. 2025Self-Composing Policies for Scalable Continual Reinforcement Learningcited by 10

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2024Theory on Mixture-of-Experts in Continual Learning
  2. 2026SPHERE: Mitigating the Loss of Spectral Plasticity in Mixture-of-Experts for Deep Reinforcement Learning
  3. 2026Dynamic Mixture of Latent Memories for Self-Evolving Agents
  4. 2026CP-MoE: Consistency-Preserving Mixture-of-Experts for Continual Learning
  5. 2025Plasticity-Aware Mixture of Experts for Learning Under QoE Shifts in Adaptive Video Streaming
  6. 2026DIMoE-Adapters: Dynamic Expert Evolution for Continual Learning in Vision-Language Models
  7. 2026NeuCME: Toward Dynamic Multimodal Continual Learning via Neural Combinatorics of Multiple Experts
  8. 2024Mixtures of Experts Unlock Parameter Scaling for Deep RL