mlsanityData to 7 Oct 2026

Mastering Massive Multi-Task Reinforcement Learning via Mixture-of-Expert Decision Transformer

arXiv 2505.24378 ↗First posted 30 May 2025cs.LG cs.AIcited by 15

On the map: Mixture-of-experts → MoE in reinforcement learning and robotics

also in 1 more
Abstract (arXiv)

Despite recent advancements in offline multi-task reinforcement learning (MTRL) have harnessed the powerful capabilities of the Transformer architecture, most approaches focus on a limited number of tasks, with scaling to extremely massive tasks remaining a formidable challenge. In this paper, we first revisit the key impact of task numbers on current MTRL method, and further reveal that naively expanding the parameters proves insufficient to counteract the performance degradation as the number of tasks escalates. Building upon these insights, we propose M3DT, a novel mixture-of-experts (MoE) framework that tackles task scalability by further unlocking the model's parameter scalability. Specifically, we enhance both the architecture and the optimization of the agent, where we strengthen the Decision Transformer (DT) backbone with MoE to reduce task load on parameter subsets, and introduce a three-stage training mechanism to facilitate efficient training with optimal performance. Experimental results show that, by increasing the number of experts, M3DT not only consistently enhances its performance as model expansion on the fixed task numbers, but also exhibits remarkable task scalability, successfully extending to 160 tasks with superior performance.

Key facts. Cited by 15 papers in the map (data through 7 Oct 2026). Placed in MoE in reinforcement learning and robotics, one of 12 directions in Mixture-of-experts. Builds on 37 of its 41 references that are in the map. First posted on arXiv 30 May 2025.

Builds on

37 of 41 references.

Closest ancestors · 2

Same direction, newest first

  1. 2024MENTOR: Mixture-of-Experts Network with Task-Oriented Perturbation for Visual Reinforcement Learning19 Oct
  2. 2024Mixtures of Experts Unlock Parameter Scaling for Deep RL13 Feb

Other references · 35

  1. 2017Attention Is All You Needcited by 59,932
  2. 2020Language Models are Few-Shot Learnerscited by 32,199
  3. 2020Scaling Laws for Neural Language Modelscited by 6,411
  4. 2017Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networkscited by 4,861
  5. 2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,635
  6. 2017Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layercited by 3,413
  7. 2021Switch Transformers: Scaling to Trillion Parameter Models with Simple and Efficient Sparsitycited by 3,014
  8. 2021Pre-train, Prompt, and Predict: A Systematic Survey of Prompting Methods in Natural Language Processingcited by 2,222
  9. 2020Offline Reinforcement Learning: Tutorial, Review, and Perspectives on Open Problemscited by 1,873
  10. 2020GShard: Scaling Giant Models with Conditional Computation and Automatic Shardingcited by 1,601
  11. 2021Decision Transformer: Reinforcement Learning via Sequence Modelingcited by 1,528
  12. 2020Gradient Surgery for Multi-Task Learningcited by 1,210
23 more
  1. 2019Meta-World: A Benchmark and Evaluation for Multi-Task and Meta Reinforcement Learningcited by 1,168
  2. 2018DeepMind Control Suitecited by 935
  3. 2022A Generalist Agentcited by 802
  4. 2021Scaling Vision with Sparse Mixture of Expertscited by 740
  5. 2021Offline Reinforcement Learning as One Big Sequence Modeling Problemcited by 636
  6. 2023TD-MPC2: Scalable, Robust World Models for Continuous Controlcited by 482
  7. 2021Conflict-Averse Gradient Descent for Multi-task Learningcited by 441
  8. 2022Multi-Game Decision Transformerscited by 212
  9. 2022Multimodal Contrastive Learning with LIMoE: the Language-Image Mixture of Expertscited by 202
  10. 2020Just Pick a Sign: Optimizing Deep Multitask Models with Gradient Sign Dropoutcited by 199
  11. 2022Prompting Decision Transformer for Few-Shot Policy Generalizationcited by 143
  12. 2020Multi-Task Reinforcement Learning with Soft Modularizationcited by 138
  13. 2023Diffusion Model is an Effective Planner and Data Synthesizer for Multi-Task Reinforcement Learningcited by 114
  14. 2022M3ViT: Mixture-of-Experts Vision Transformer for Efficient Multi-task Learning with Model-Accelerator Co-designcited by 97
  15. 2018ProMP: Proximal Meta-Policy Searchcited by 91
  16. 2024Merging Multi-Task Models via Weight-Ensembling Mixture of Expertscited by 85
  17. 2023TaskExpert: Dynamically Assembling Multi-Task Representations with Memorial Mixture-of-Expertscited by 45
  18. 2023Multi-Task Reinforcement Learning with Mixture of Orthogonal Expertscited by 45
  19. 2024Sharing Knowledge in Multi-Task Deep Reinforcement Learningcited by 42
  20. 2022PaCo: Parameter-Compositional Multi-Task Reinforcement Learningcited by 40
  21. 2020Continuous Action Reinforcement Learning from a Mixture of Interpretable Expertscited by 17
  22. 2024HarmoDT: Harmony Multi-Task Decision Transformer for Offline Reinforcement Learningcited by 14
  23. 2024QPO: Query-dependent Prompt Optimization via Multi-Loop Offline Reinforcement Learningcited by 10

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 2; 2026: 13.

15 papers.

Sort:

By area:

  1. 2025Mixture-of-Experts Meets In-Context Reinforcement Learningcited by 7
  2. 2026ACE-Brain-0.5: A Unified Embodied Foundational Model for Physical Agentic AIcited by 5
  3. 2026DyGRO-VLA: Cross-Task Scaling of Vision-Language-Action Models via Dynamic Grouped Residual Optimizationcited by 2
  4. 2026ExToken: Structured Exploration for Efficient Vision-Language-Action Reinforcement Fine-tuningcited by 2
  5. 2025Guided by the Experts: Provable Feature Learning Dynamic of Soft-Routed Mixture-of-Expertscited by 1
  6. 2026Mixture-of-World Models: Scaling Multi-Task Reinforcement Learning with Modular Latent Dynamicscited by 1
  7. 2026Generalizable Multi-Task Learning for Wireless Networks Using Prompt Decision Transformerscited by 1
  8. 2026Conditional Sequence Modeling for Safe Reinforcement Learningcited by 0
  9. 2026CORE-MTL: Rethinking Gradient Balancing via Causal Orthogonal Representationscited by 0
  10. 2026Representation Learning Enables Scalable Multitask Deep Reinforcement Learningcited by 0
  11. 2026CTS-MoE: Implicit Terrain Adaptation via Mixture-of-Experts for Perceptive Locomotioncited by 0
  12. 2026Beyond Isolation: Unlocking Reinforcement Learning Component Synergy for Sample-Efficient Continuous Controlcited by 0
3 more
  1. 2026WiSDoM: Wireless Sparse Decision Transformer with Mixture-of-Experts for Multi-Task Mobile Network Optimizationcited by 0
  2. 2026Hierarchical Wasserstein Merging for Multi-Domain Multi-Task Learning: From Specialists to a Generalistcited by 0
  3. 2026DaCe-DT: Data-Centric Offline Multi-Task Reinforcement Learning via Adaptive Prompts and Trajectory Correction for Heterogeneous Taskscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Towards Stable and Effective Reinforcement Learning for Mixture-of-Experts
  2. 2024Task-Aware Harmony Multi-Task Decision Transformer for Offline Reinforcement Learning
  3. 2025One Model for All Tasks: Leveraging Efficient World Models in Multi-Task Planning
  4. 2026Phase-Aware Mixture of Experts for Agentic Reinforcement Learning
  5. 2026RoutePack: Expert Placement and Attention-Aware Data Packing for MoE Reinforcement Learning
  6. 2026Vintix II: Decision Pre-Trained Transformer is a Scalable In-Context Reinforcement Learner
  7. 2025Knowledge Transfer in Model-Based Reinforcement Learning Agents for Efficient Multi-Task Learning
  8. 2024Skills Regularized Task Decomposition for Multi-task Offline Reinforcement Learning