mlsanityData to 14 Oct 2026

Multi-Task Reinforcement Learning with Mixture of Orthogonal Experts

arXiv 2311.11385 ↗First posted 19 Nov 2023cs.LGcited by 49

Not on the map. Why?

Cited by 26 in Deep reinforcement learning methods, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Multi-Task Reinforcement Learning (MTRL) tackles the long-standing problem of endowing agents with skills that generalize across a variety of problems. To this end, sharing representations plays a fundamental role in capturing both unique and common characteristics of the tasks. Tasks may exhibit similarities in terms of skills, objects, or physical properties while leveraging their representations eases the achievement of a universal policy. Nevertheless, the pursuit of learning a shared set of diverse representations is still an open challenge. In this paper, we introduce a novel approach for representation learning in MTRL that encapsulates common structures among the tasks using orthogonal representations to promote diversity. Our method, named Mixture Of Orthogonal Experts (MOORE), leverages a Gram-Schmidt process to shape a shared subspace of representations generated by a mixture of experts. When task-specific information is provided, MOORE generates relevant representations from this shared subspace. We assess the effectiveness of our approach on two MTRL benchmarks, namely MiniGrid and MetaWorld, showing that MOORE surpasses related baselines and establishes a new state-of-the-art result on MetaWorld.

Key facts. Cited by 49 papers in the map (data through 14 Oct 2026). Builds on 26 of its 38 references that are in the map. First posted on arXiv 19 Nov 2023.

Builds on

26 of 38 references.

References · 26

  1. 2017Proximal Policy Optimization Algorithmscited by 15,215
  2. 2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,885
  3. 2013Playing Atari with Deep Reinforcement Learningcited by 3,274
  4. 2015Trust Region Policy Optimizationcited by 2,580
  5. 2017FiLM: Visual Reasoning with a General Conditioning Layercited by 2,333
  6. 2020Gradient Surgery for Multi-Task Learningcited by 1,256
  7. 2019Meta-World: A Benchmark and Evaluation for Multi-Task and Meta Reinforcement Learningcited by 1,204
  8. 2017Rainbow: Combining Improvements in Deep Reinforcement Learningcited by 786
  9. 2018Diversity is All You Need: Learning Skills without a Reward Functioncited by 752
  10. 2017Mastering Chess and Shogi by Self-Play with a General Reinforcement Learning Algorithmcited by 734
  11. 2023Minigrid & Miniworld: Modular & Customizable Reinforcement Learning Environments for Goal-Oriented Taskscited by 327
  12. 2017Distral: Robust Multitask Reinforcement Learningcited by 233
14 more
  1. 2021Multi-Task Reinforcement Learning with Context-based Representationscited by 174
  2. 2020Multi-Task Reinforcement Learning with Soft Modularizationcited by 143
  3. 2018Multi-task Deep Reinforcement Learning with PopArtcited by 141
  4. 2016Learning Modular Neural Network Policies for Multi-Task and Multi-Robot Transfercited by 127
  5. 2020Continual Learning in Low-rank Orthogonal Subspacescited by 99
  6. 2017Orthogonal Weight Normalization: Solution to Optimization over Multiple Dependent Stiefel Manifolds in Deep Neural Networkscited by 92
  7. 2020Efficient Riemannian Optimization on the Stiefel Manifold via the Cayley Transformcited by 80
  8. 2019Orthogonal Deep Neural Networkscited by 72
  9. 2020MushroomRL: Simplifying Reinforcement Learning Researchcited by 50
  10. 2022PaCo: Parameter-Compositional Multi-Task Reinforcement Learningcited by 46
  11. 2017Orthogonal Machine Learning: Power and Limitationscited by 28
  12. 2021Probabilistic Mixture-of-Experts for Efficient Deep Reinforcement Learningcited by 27
  13. 2020Continuous Action Reinforcement Learning from a Mixture of Interpretable Expertscited by 17
  14. 2016Optimization on Submanifolds of Convolution Kernels in CNNscited by 15

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 7; 2025: 22; 2026: 20.

49 papers.

Sort:

By area:

  1. 2025A Comprehensive Survey of Mixture-of-Experts: Algorithms, Theory, and Applicationscited by 91
  2. 2024Mixtures of Experts Unlock Parameter Scaling for Deep RLcited by 78
  3. 2024One Policy to Run Them All: an End-to-end Learning Approach to Multi-Embodiment Locomotioncited by 43
  4. 2025Meta-World+: An Improved, Standardized, RL Benchmarkcited by 43
  5. 2025Advancing Expert Specialization for Better MoEcited by 39
  6. 2025Load Balancing Mixture of Experts with Similarity Preserving Routerscited by 22
  7. 2024Mixture of Experts in a Mixture of RL settingscited by 21
  8. 2024Variational Distillation of Diffusion Policies into Mixture of Expertscited by 15
  9. 2025Mastering Massive Multi-Task Reinforcement Learning via Mixture-of-Expert Decision Transformercited by 15
  10. 2024Acquiring Diverse Skills using Curriculum Reinforcement Learning with Mixture of Expertscited by 14
  11. 2025KungfuBot2: Learning Versatile Motion Skills for Humanoid Whole-Body Controlcited by 13
  12. 2025Benchmarking Massively Parallelized Multi-Task Reinforcement Learning for Robotics Taskscited by 11
37 more
  1. 2025Coupling Experts and Routers in Mixture-of-Experts via an Auxiliary Losscited by 11
  2. 2024Active Fine-Tuning of Multi-Task Policiescited by 9
  3. 2025Multi-Task Reinforcement Learning Enables Parameter Scalingcited by 8
  4. 2025One Subgoal at a Time: Zero-Shot Generalization to Arbitrary Linear Temporal Logic Requirements in Multi-Task Reinforcement Learningcited by 8
  5. 2024Model-Based Transfer Learning for Contextual Reinforcement Learningcited by 7
  6. 2025Learning Policy Committees for Effective Personalization in MDPs with Diverse Taskscited by 4
  7. 2025Plasticity-Aware Mixture of Experts for Learning Under QoE Shifts in Adaptive Video Streamingcited by 4
  8. 2025Bridging the Performance Gap Between Target-Free and Target-Based Reinforcement Learningcited by 4
  9. 2025Interleaved Multitask Learning with Energy Modulated Learning Progresscited by 3
  10. 2025One Model for All Tasks: Leveraging Efficient World Models in Multi-Task Planningcited by 3
  11. 2026Continual Policy Consolidation for Lifelong Robot Learningcited by 3
  12. 2025Sparsity-Driven Plasticity in Multi-Task Reinforcement Learningcited by 2
  13. 2025Prismatic World Model: Learning Compositional Dynamics for Planning in Hybrid Systemscited by 2
  14. 2026AnchorMoE: Interpretable Time Series Classification via Anchor-Routed MoEcited by 2
  15. 2025Nonparametric Bellman Mappings for Value Iteration in Distributed Reinforcement Learningcited by 1
  16. 2025Policy Search, Retrieval, and Composition via Task Similarity in Collaborative Agentic Systemscited by 1
  17. 2026Mixture-of-World Models: Scaling Multi-Task Reinforcement Learning with Modular Latent Dynamicscited by 1
  18. 2026STAIRS-Former: Spatio-Temporal Attention with Interleaved Recursive Structure Transformer for Offline Multi-task Multi-agent Reinforcement Learningcited by 1
  19. 2026FLAME: Adaptive Mixture-of-Experts for Continual Multimodal Multi-Task Learningcited by 1
  20. 2026Building2Building: A Large Scale Benchmark for Generalizable Real-World Reinforcement Learningcited by 1
  21. 2025Multitask Reinforcement Learning for Quadcopter Attitude Stabilization and Tracking using Graph Policycited by 0
  22. 2025Time Reversal Symmetry for Efficient Robotic Manipulations in Deep Reinforcement Learningcited by 0
  23. 2025Leveraging Temporally Extended Behavior Sharing for Multi-task Reinforcement Learningcited by 0
  24. 2026Diverse Skill Discovery for Quadruped Robots via Unsupervised Learningcited by 0
  25. 2026Robust Exploration in Directed Controller Synthesis via Reinforcement Learning with Soft Mixture-of-Expertscited by 0
  26. 2026Hierarchical Lead Critic based Multi-Agent Reinforcement Learningcited by 0
  27. 2026Diffusion Policy with Bayesian Expert Selection for Active Multi-Target Trackingcited by 0
  28. 2026TOPPO: Rethinking PPO for Multi-Task Reinforcement Learning with Critic Balancingcited by 0
  29. 2026Distributionally Robust Multi-Task Reinforcement Learning via Adaptive Task Samplingcited by 0
  30. 2026Learning in Low-Dimensional Subspaces: Orthogonal Bottlenecks for Reinforcement Learningcited by 0
  31. 2026CORE-MTL: Rethinking Gradient Balancing via Causal Orthogonal Representationscited by 0
  32. 2026Formalizing Task-Space Complexity for Zero-Shot Generalizationcited by 0
  33. 2026Learning Subset-Shared Invariances for Domain Generalization with Mixture-of-Expertscited by 0
  34. 2026Learning Adaptive Multi-Task Guidance, Navigation, and Control via Hypernetworkscited by 0
  35. 2026Task Specialization Fine-Tuning for Contextual Reinforcement Learningcited by 0
  36. 2026ElectrolyteFM: Unifying Electrolyte Property Prediction through Cross-Property Knowledge Learningcited by 0
  37. 2026DaCe-DT: Data-Centric Offline Multi-Task Reinforcement Learning via Adaptive Prompts and Trajectory Correction for Heterogeneous Taskscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026Provable Multi-Task Reinforcement Learning: A Representation Learning Framework with Low Rank Rewards
  2. 2025Mixture-of-Experts Meets In-Context Reinforcement Learning
  3. 2024Sharing Knowledge in Multi-Task Deep Reinforcement Learning
  4. 2022Mod-Squad: Designing Mixture of Experts As Modular Multi-Task Learners
  5. 2025TIMRL: A Novel Meta-Reinforcement Learning Framework for Non-Stationary and Multi-Task Environments
  6. 2022Provable Benefit of Multitask Representation Learning in Reinforcement Learning
  7. 2025Adaptive Shared Experts with LoRA-Based Mixture of Experts for Multi-Task Learning
  8. 2024The Power of Active Multi-Task Learning in Reinforcement Learning from Human Feedback