mlsanityData to 7 Oct 2026

Acquiring Diverse Skills using Curriculum Reinforcement Learning with Mixture of Experts

arXiv 2403.06966 ↗First posted 11 Mar 2024cs.LG cs.ROcited by 14

On the map: Mixture-of-experts → MoE in reinforcement learning and robotics

Abstract (arXiv)

Reinforcement learning (RL) is a powerful approach for acquiring a good-performing policy. However, learning diverse skills is challenging in RL due to the commonly used Gaussian policy parameterization. We propose Diverse Skill Learning (Di-SkilLfootnoteVideos and code are available on the project webpage: urlhttps://alrhub.github.io/di-skill-website/), an RL method for learning diverse skills using Mixture of Experts, where each expert formalizes a skill as a contextual motion primitive. Di-SkilL optimizes each expert and its associate context distribution to a maximum entropy objective that incentivizes learning diverse skills in similar contexts. The per-expert context distribution enables automatic curricula learning, allowing each expert to focus on its best-performing sub-region of the context space. To overcome hard discontinuities and multi-modalities without any prior knowledge of the environment's unknown context probability space, we leverage energy-based models to represent the per-expert context distributions and demonstrate how we can efficiently train them using the standard policy gradient objective. We show on challenging robot simulation tasks that Di-SkilL can learn diverse and performant skills.

Key facts. Cited by 14 papers in the map (data through 7 Oct 2026). Placed in MoE in reinforcement learning and robotics, one of 12 directions in Mixture-of-experts. Builds on 41 of its 64 references that are in the map. First posted on arXiv 11 Mar 2024.

Builds on

41 of 64 references.

Closest ancestors · 2

Same direction, newest first

  1. 2021Specializing Versatile Skill Libraries using Local Mixture of Experts8 Dec
  2. 2021Probabilistic Mixture-of-Experts for Efficient Deep Reinforcement Learning19 Apr

Other references · 39

  1. 2017Proximal Policy Optimization Algorithmscited by 14,441
  2. 2023Diffusion Policy: Visuomotor Policy Learning via Action Diffusioncited by 2,730
  3. 2015Trust Region Policy Optimizationcited by 2,503
  4. 2019Normalizing Flows for Probabilistic Modeling and Inferencecited by 1,315
  5. 2021Deep Reinforcement Learning at the Edge of the Statistical Precipicecited by 781
  6. 2018Diversity is All You Need: Learning Skills without a Reward Functioncited by 734
  7. 2021Implicit Behavioral Cloningcited by 442
  8. 2022Behavior Transformers: Cloning k modes with one stonecited by 299
  9. 2020Emergent Complexity and Zero-shot Transfer via Unsupervised Environment Designcited by 260
  10. 2017Automatic Goal Generation for Reinforcement Learning Agentscited by 252
  11. 2014Robots that can adapt like animalscited by 203
  12. 2020Prioritized Level Replaycited by 190
27 more
  1. 2019Efficient Exploration via State Marginal Matchingcited by 186
  2. 2017Reverse Curriculum Generation for Reinforcement Learningcited by 185
  3. 2020Implementation Matters in Deep Policy Gradients: A Case Study on PPO and TRPOcited by 184
  4. 2017Intrinsic Motivation and Automatic Curricula via Asymmetric Self-Playcited by 139
  5. 2021URLB: Unsupervised Reinforcement Learning Benchmarkcited by 138
  6. 2020Explore, Discover and Learn: Unsupervised Discovery of State-Covering Skillscited by 120
  7. 2021APS: Active Pretraining with Successor Featurescited by 109
  8. 2021Replay-Guided Adversarial Environment Designcited by 108
  9. 2020Automatic Curriculum Learning through Value Disagreementcited by 86
  10. 2024Towards Diverse Behaviors: A Benchmark for Imitation Learning with Human Demonstrationscited by 51
  11. 2020Self-Paced Deep Reinforcement Learningcited by 49
  12. 2023Multi-Task Reinforcement Learning with Mixture of Orthogonal Expertscited by 45
  13. 2021The Information Geometry of Unsupervised Reinforcement Learningcited by 36
  14. 2019Self-Paced Contextual Reinforcement Learningcited by 26
  15. 2022ProDMPs: A Unified Perspective on Dynamic and Probabilistic Movement Primitivescited by 25
  16. 2021Differentiable Trust Region Layers for Deep Reinforcement Learningcited by 24
  17. 2023MAP-Elites with Descriptor-Conditioned Gradients and Archive Distillation into a Single Policycited by 20
  18. 2023Reparameterized Policy Learning for Multimodal Trajectory Optimizationcited by 20
  19. 2022Deep Black-Box Reinforcement Learning with Movement Primitivescited by 18
  20. 2020Continuous Action Reinforcement Learning from a Mixture of Interpretable Expertscited by 17
  21. 2020Model-Based Quality-Diversity Search for Efficient Robot Learningcited by 14
  22. 2024Open the Black Box: Step-based Policy Updates for Temporally-Correlated Episodic Reinforcement Learningcited by 12
  23. 2022Hierarchical Policy Blending as Inference for Reactive Robot Controlcited by 8
  24. 2023On the Benefit of Optimal Transport for Curriculum Reinforcement Learningcited by 6
  25. 2016Policy Search with High-Dimensional Context Variablescited by 5
  26. 2020Contextual Latent-Movements Off-Policy Optimization for Robotic Manipulation Skillscited by 5
  27. 2022Hierarchical Policy Blending As Optimal Transportcited by 4

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 5; 2025: 8; 2026: 1.

14 papers.

Sort:

By area:

  1. 2025A Comprehensive Survey of Mixture-of-Experts: Algorithms, Theory, and Applicationscited by 91
  2. 2024Efficient Diffusion Transformer Policies with Mixture of Expert Denoisers for Multitask Learningcited by 46
  3. 2025Diversity-Incentivized Exploration for Versatile Reasoningcited by 28
  4. 2024TOP-ERL: Transformer-based Off-Policy Episodic Reinforcement Learningcited by 18
  5. 2025MoE-Loco: Mixture of Experts for Multitask Locomotioncited by 17
  6. 2024Variational Distillation of Diffusion Policies into Mixture of Expertscited by 14
  7. 2025Geometry-aware RL for Manipulation of Varying Shapes and Deformable Objectscited by 10
  8. 2024Efficient Residual Learning with Mixture-of-Experts for Universal Dexterous Graspingcited by 8
  9. 2025TROLL: Trust Regions improve Reinforcement Learning for Large Language Modelscited by 7
  10. 2025MoPFormer: Motion-Primitive Transformer for Wearable-Sensor Activity Recognitioncited by 5
  11. 2025Adaptive Shielding for Safe Reinforcement Learning under Hidden-Parameter Dynamics Shiftscited by 2
  12. 2024Skill-Enhanced Reinforcement Learning Acceleration from Heterogeneous Demonstrationscited by 1
2 more
  1. 2025Trajectory First: A Curriculum for Discovering Diverse Policiescited by 0
  2. 2026Self Paced Gaussian Contextual Reinforcement Learningcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026Learning Hierarchical Skill Policies with Offline Quality-Diversity Reinforcement Learning
  2. 2023Information Maximizing Curriculum: A Curriculum-Based Approach for Imitating Diverse Skills
  3. 2023Offline Diversity Maximization Under Imitation Constraints
  4. 2024Robust Policy Learning via Offline Skill Diffusion
  5. 2026DSD: Learning Diverse and Reusable Motor Skills via Diffusion Skill Discovery
  6. 2022The Expertise Problem: Learning from Specialized Feedback
  7. 2021Robot Skill Adaptation via Soft Actor-Critic Gaussian Mixture Models
  8. 2025cMALC-D: Contextual Multi-Agent LLM-Guided Curriculum Learning with Diversity-Based Context Blending