mlsanityData to 7 Oct 2026

Nested Mixture of Experts: Cooperative and Competitive Learning of Hybrid Dynamical System

arXiv 2011.10605 ↗First posted 20 Nov 2020cs.LG cs.ROcited by 3

On the map: Mixture-of-experts → MoE in reinforcement learning and robotics

Abstract (arXiv)

Model-based reinforcement learning (MBRL) algorithms can attain significant sample efficiency but require an appropriate network structure to represent system dynamics. Current approaches include white-box modeling using analytic parameterizations and black-box modeling using deep neural networks. However, both can suffer from a bias-variance trade-off in the learning process, and neither provides a structured method for injecting domain knowledge into the network. As an alternative, gray-box modeling leverages prior knowledge in neural network training but only for simple systems. In this paper, we devise a nested mixture of experts (NMOE) for representing and learning hybrid dynamical systems. An NMOE combines both white-box and black-box models while optimizing bias-variance trade-off. Moreover, an NMOE provides a structured method for incorporating various types of prior knowledge by training the associative experts cooperatively or competitively. The prior knowledge includes information on robots' physical contacts with the environments as well as their kinematic and dynamic properties. In this paper, we demonstrate how to incorporate prior knowledge into our NMOE in various continuous control domains, including hybrid dynamical systems. We also show the effectiveness of our method in terms of data-efficiency, generalization to unseen data, and bias-variance trade-off. Finally, we evaluate our NMOE using an MBRL setup, where the model is integrated with a model-based controller and trained online.

Key facts. Cited by 3 papers in the map (data through 7 Oct 2026). Placed in MoE in reinforcement learning and robotics, one of 12 directions in Mixture-of-experts. Builds on 14 of its 21 references that are in the map. First posted on arXiv 20 Nov 2020.

Builds on

14 of 21 references.

References · 14

  1. 2017Proximal Policy Optimization Algorithmscited by 14,441
  2. 2018Addressing Function Approximation Error in Actor-Critic Methodscited by 2,302
  3. 2018DeepMind Control Suitecited by 935
  4. 2018Deep Reinforcement Learning in a Handful of Trials using Probabilistic Dynamics Modelscited by 748
  5. 2019Hamiltonian Neural Networkscited by 702
  6. 2017Neural Network Dynamics for Model-Based Deep Reinforcement Learning with Model-Free Fine-Tuningcited by 288
  7. 2019Deep Lagrangian Networks: Using Physics as Model Prior for Deep Learningcited by 205
  8. 2019Symplectic ODE-Net: Learning Hamiltonian Dynamics with Controlcited by 183
  9. 2018Multistep Neural Networks for Data-driven Discovery of Nonlinear Dynamical Systemscited by 88
  10. 2020Encoding Physical Constraints in Differentiable Newton-Euler Algorithmcited by 23
  11. 2020Structured Mechanical Models for Robot Learning and Controlcited by 21
  12. 2019Learning-based Model Predictive Control for Safe Exploration and Reinforcement Learningcited by 16
2 more
  1. 2019Control of A High Performance Bipedal Robot using Viscoelastic Liquid Cooled Actuatorscited by 1
  2. 2019Data-Efficient and Safe Learning for Humanoid Locomotion Aided by a Dynamic Balancing Modelcited by 1

Built on by

Cited by year

Citing papers per year: 2022: 1; 2023: 0; 2024: 0; 2025: 2.

3 papers.

  1. 2025Multi-Agent Collaboration Mechanisms: A Survey of LLMscited by 424
  2. 2025Prismatic World Model: Learning Compositional Dynamics for Planning in Hybrid Systemscited by 2
  3. 2022Sample Efficient Dynamics Learning for Symmetrical Legged Robots:Leveraging Physics Invariance and Geometric Symmetriescited by 1

Similar, not cited

Close in wording, no citation either way

  1. 2024Fusion Dynamical Systems with Machine Learning in Imitation Learning: A Comprehensive Overview
  2. 2020Hierarchical Decomposition of Nonlinear Dynamics and Control for System Identification and Policy Distillation
  3. 2021Model-Based Reinforcement Learning via Stochastic Hybrid Models
  4. 2019MANGA: Method Agnostic Neural-policy Generalization and Adaptation ↗
  5. 2023Multi-Task Reinforcement Learning with Mixture of Orthogonal Experts
  6. 2025Expertise need not monopolize: Action-Specialized Mixture of Experts for Vision-Language-Action Learning
  7. 2020Contextual Policy Transfer in Reinforcement Learning Domains via Deep Mixtures-of-Experts
  8. 2024An Efficient Model-Based Approach on Learning Agile Motor Skills without Reinforcement