# MPC-Net: A First Principles Guided Policy Search

arXiv: https://arxiv.org/abs/1909.05197 · HTML: https://mlsanity.com/paper/1909.05197/

## Key facts

- Cited by 34 papers in the map (data through 14 Oct 2026).
- Placed in Imitation learning methods, one of 13 directions in Offline RL and imitation learning from data.
- First posted on arXiv 11 Sep 2019.

## Placed in

- Offline RL and imitation learning from data / [Imitation learning methods](https://mlsanity.com/offlinerl/imitation-learning-methods/)

## Built on by (most cited first, 30 of 34)

- [Contraction Theory for Nonlinear Stability Analysis and Learning-based Control: A Tutorial Overview](https://mlsanity.com/paper/2110.00675/) (2021)
- [Imitate and Repurpose: Learning Reusable Robot Movement Skills From Human and Animal Behaviors](https://mlsanity.com/paper/2203.17138/) (2022)
- [Learning Free Gait Transition for Quadruped Robots via Phase-Guided Controller](https://mlsanity.com/paper/2201.00206/) (2022)
- [Synthesis of Model Predictive Control and Reinforcement Learning: Survey and Classification](https://mlsanity.com/paper/2502.02133/) (2025)
- [GLiDE: Generalizable Quadrupedal Locomotion in Diverse Environments with a Centroidal Model](https://mlsanity.com/paper/2104.09771/) (2021)
- [Imitation Learning from MPC for Quadrupedal Multi-Gait Control](https://mlsanity.com/paper/2103.14331/) (2021)
- [Evaluating model-based planning and planner amortization for continuous control](https://mlsanity.com/paper/2110.03363/) (2021)
- [Rethinking Robustness Assessment: Adversarial Attacks on Learning-based Quadrupedal Locomotion Controllers](https://mlsanity.com/paper/2405.12424/) (2024)
- [Learning-based Robust Motion Planning with Guaranteed Stability: A Contraction Theory Approach](https://mlsanity.com/paper/2102.12668/) (2021)
- [Demonstration-Efficient Guided Policy Search via Imitation of Robust Tube MPC](https://mlsanity.com/paper/2109.09910/) (2021)
- [Approximate non-linear model predictive control with safety-augmented neural networks](https://mlsanity.com/paper/2304.09575/) (2023)
- [Learning to Walk by Steering: Perceptive Quadrupedal Locomotion in Dynamic Environments](https://mlsanity.com/paper/2209.09233/) (2022)
- [Planning-Guided Diffusion Policy Learning for Generalizable Contact-Rich Bimanual Manipulation](https://mlsanity.com/paper/2412.02676/) (2024)
- [Parameter-Adaptive Approximate MPC: Tuning Neural-Network Controllers without Retraining](https://mlsanity.com/paper/2404.05835/) (2024)
- [Imitation Learning from Nonlinear MPC via the Exact Q-Loss and its Gauss-Newton Approximation](https://mlsanity.com/paper/2304.01782/) (2023)
- [Learning Coordinated Terrain-Adaptive Locomotion by Imitating a Centroidal Dynamics Planner](https://mlsanity.com/paper/2111.00262/) (2021)
- [The Mini Wheelbot: A Testbed for Learning-based Balancing, Flips, and Articulated Driving](https://mlsanity.com/paper/2502.04582/) (2025)
- [Safe Imitation Learning of Nonlinear Model Predictive Control for Flexible Robots](https://mlsanity.com/paper/2212.02941/) (2022)
- [Diffusion-Based Approximate MPC: Fast and Consistent Imitation of Multi-Modal Action Distributions](https://mlsanity.com/paper/2504.04603/) (2025)
- [Generative Models From and For Sampling-Based MPC: A Bootstrapped Approach For Adaptive Contact-Rich Manipulation](https://mlsanity.com/paper/2510.14643/) (2025)
- [CACTO-BIC: Scalable Actor-Critic Learning via Biased Sampling and GPU-Accelerated Trajectory Optimization](https://mlsanity.com/paper/2602.19699/) (2026)
- [Generalized Reinforcement Learning for Building Control using Behavioral Cloning](https://mlsanity.com/paper/2104.00123/) (2021)
- [Integrating Expert Guidance for Efficient Learning of Safe Overtaking in Autonomous Driving Using Deep Reinforcement Learning](https://mlsanity.com/paper/2308.09456/) (2023)
- [OGMP: Oracle Guided Multi-mode Policies for Agile and Versatile Robot Control](https://mlsanity.com/paper/2403.04205/) (2024)
- [MPC-Injection: Biasing Off-Policy Locomotion RL Toward Controller-Induced Behavior Basins](https://mlsanity.com/paper/2606.26392/) (2026)
- [SUN: Agentic Robot Policy Learning with Persistent Task Programs](https://mlsanity.com/paper/2608.31167/) (2026)
- [Knowledge-Integrated Informed AI for National Security](https://mlsanity.com/paper/2202.03188/) (2022)
- [Efficient Deep Learning of Robust, Adaptive Policies using Tube MPC-Guided Data Augmentation](https://mlsanity.com/paper/2303.15688/) (2023)
- [Bridging RL and MPC for mixed-integer optimal control with application to Formula 1 race strategies](https://mlsanity.com/paper/2604.00826/) (2026)
- [Neuro-Inspired Inverse Learning for Planning and Control](https://mlsanity.com/paper/2605.24152/) (2026)

## Abstract (arXiv)

We present an Imitation Learning approach for the control of dynamical systems with a known model. Our policy search method is guided by solutions from MPC. Typical policy search methods of this kind minimize a distance metric between the guiding demonstrations and the learned policy. Our loss function, however, corresponds to the minimization of the control Hamiltonian, which derives from the principle of optimality. Therefore, our algorithm directly attempts to solve the optimality conditions with a parameterized class of control laws. Additionally, the proposed loss function explicitly encodes the constraints of the optimal control problem and we provide numerical evidence that its minimization achieves improved constraint satisfaction. We train a mixture-of-expert neural network architecture for controlling a quadrupedal robot and show that this policy structure is well suited for such multimodal systems. The learned policy can successfully stabilize different gaits on the real walking robot from less than 10 min of demonstration data.