SAC-MoE: Reinforcement Learning with Mixture-of-Experts for Control of Hybrid Dynamical Systems with Uncertainty

arXiv 2511.12361 ↗First posted 15 Nov 2025cs.RO cs.SY eess.SYcited by 0

On the map: Deep reinforcement learning methods → RL theory and sample complexity

Abstract (arXiv)

Hybrid dynamical systems result from the interaction of continuous-variable dynamics with discrete events and encompass various systems such as legged robots, vehicles and aircrafts. Challenges arise when the system's modes are characterized by unobservable (latent) parameters and the events that cause system dynamics to switch between different modes are also unobservable. Model-based control approaches typically do not account for such uncertainty in the hybrid dynamics, while standard model-free RL methods fail to account for abrupt mode switches, leading to poor generalization. To overcome this, we propose SAC-MoE which models the actor of the Soft Actor-Critic (SAC) framework as a Mixture-of-Experts (MoE) with a learned router that adaptively selects among learned experts. To further improve robustness, we develop a curriculum-based training algorithm to prioritize data collection in challenging settings, allowing better generalization to unseen modes and switching locations. Simulation studies in hybrid autonomous racing and legged locomotion tasks show that SAC-MoE outperforms baselines (up to 6x) in zero-shot generalization to unseen environments. Our curriculum strategy consistently improves performance across all evaluated policies. Qualitative analysis shows that the interpretable MoE router activates different experts for distinct latent modes.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in RL theory and sample complexity, one of 15 directions in Deep reinforcement learning methods. Builds on 19 of its 33 references that are in the map. First posted on arXiv 15 Nov 2025.

Builds on

19 of 33 references.

References · 19

2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,885
2017Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layercited by 3,509
2018Soft Actor-Critic Algorithms and Applicationscited by 1,240
2018Data-Efficient Hierarchical Reinforcement Learningcited by 455
2016The Option-Critic Architecturecited by 448
2020Curriculum Learning for Reinforcement Learning Domains: A Framework and Surveycited by 373
2015Contextual Markov Decision Processescited by 220
2019Active Domain Randomizationcited by 170
2022Rapid Locomotion via Reinforcement Learningcited by 168
2017Preparing for the Unknown: Learning a Universal Policy with Online System Identificationcited by 141
2021RL for Latent MDPs: Regret Guarantees and a Lower Boundcited by 70
2022Contextualize Me -- The Case for Context in Reinforcement Learningcited by 46

7 more

2023AMAGO: Scalable In-Context Reinforcement Learning for Adaptive Agentscited by 41
2024Multi-Head Mixture-of-Expertscited by 31
2021Probabilistic Mixture-of-Experts for Efficient Deep Reinforcement Learningcited by 27
2021Neural Hybrid Automata: Learning Dynamics with Multiple Modes and Stochastic Transitionscited by 7
2025Discrete-Time Hybrid Automata Learning: Legged Locomotion Meets Skateboardingcited by 7
2024Inferring Behavior-Specific Context Improves Zero-Shot Generalization in Reinforcement Learningcited by 5
2025Tractable Stochastic Hybrid Model Predictive Control using Gaussian Processes for Repetitive Tasks in Unseen Environmentscited by 1

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way