# Learning Deep Control Policies for Autonomous Aerial Vehicles with MPC-Guided Policy Search

arXiv: https://arxiv.org/abs/1509.06791 · HTML: https://mlsanity.com/paper/1509.06791/

## Key facts

- Cited by 42 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 22 Sep 2015.

## Built on by (most cited first, 30 of 42)

- [Holo-Dex: Teaching Dexterity with Immersive Mixed Reality](https://mlsanity.com/paper/2210.06463/) (2022)
- [OmniDrones: An Efficient and Flexible Platform for Reinforcement Learning in Drone Control](https://mlsanity.com/paper/2309.12825/) (2023)
- [Lyapunov-stable Neural Control for State and Output Feedback: A Novel Formulation](https://mlsanity.com/paper/2404.07956/) (2024)
- [Better than Your Teacher: LLM Agents that learn from Privileged AI Feedback](https://mlsanity.com/paper/2410.05434/) (2024)
- [Learning Perception-Aware Agile Flight in Cluttered Environments](https://mlsanity.com/paper/2210.01841/) (2022)
- [CGD: Constraint-Guided Diffusion Policies for UAV Trajectory Planning](https://mlsanity.com/paper/2405.01758/) (2024)
- [Policy Search for Model Predictive Control with Application to Agile Drone Flight](https://mlsanity.com/paper/2112.03850/) (2021)
- [Pervasive AI for IoT applications: A Survey on Resource-efficient Distributed Artificial Intelligence](https://mlsanity.com/paper/2105.01798/) (2021)
- [Bootstrapped Model Predictive Control](https://mlsanity.com/paper/2503.18871/) (2025)
- [Optimal Cost Design for Model Predictive Control](https://mlsanity.com/paper/2104.11353/) (2021)
- [Reliably-stabilizing piecewise-affine neural network controllers](https://mlsanity.com/paper/2111.07183/) (2021)
- [Optimizing Crop Management with Reinforcement Learning and Imitation Learning](https://mlsanity.com/paper/2209.09991/) (2022)
- [Learning Low-Frequency Motion Control for Robust and Dynamic Robot Locomotion](https://mlsanity.com/paper/2209.14887/) (2022)
- [Demonstration-Efficient Guided Policy Search via Imitation of Robust Tube MPC](https://mlsanity.com/paper/2109.09910/) (2021)
- [Interval Reachability of Nonlinear Dynamical Systems with Neural Network Controllers](https://mlsanity.com/paper/2301.07912/) (2023)
- [Efficient Interaction-Aware Interval Analysis of Neural Network Feedback Loops](https://mlsanity.com/paper/2307.14938/) (2023)
- [Neural Lyapunov Differentiable Predictive Control](https://mlsanity.com/paper/2205.10728/) (2022)
- [Two-Stage Learning of Stabilizing Neural Controllers via Zubov Sampling and Iterative Domain Expansion](https://mlsanity.com/paper/2506.01356/) (2025)
- [Handling Sparse Rewards in Reinforcement Learning Using Model Predictive Control](https://mlsanity.com/paper/2210.01525/) (2022)
- [Neuromorphic Attitude Estimation and Control](https://mlsanity.com/paper/2411.13945/) (2024)
- [Reinforcement Learning using Guided Observability](https://mlsanity.com/paper/2104.10986/) (2021)
- [Constrained Imitation Learning for a Flapping Wing Unmanned Aerial Vehicle](https://mlsanity.com/paper/2206.03987/) (2022)
- [Accelerating and Scaling MPC-Guided Reinforcement Learning for Humanoid Locomotion and Manipulation](https://mlsanity.com/paper/2606.05687/) (2026)
- [Efficient Motion Planning for Automated Lane Change based on Imitation Learning and Mixed-Integer Optimization](https://mlsanity.com/paper/1904.08784/) (2019)
- [Learning to Execute: Efficient Learning of Universal Plan-Conditioned Policies in Robotics](https://mlsanity.com/paper/2111.07908/) (2021)
- [Output Feedback Tube MPC-Guided Data Augmentation for Robust, Efficient Sensorimotor Policy Learning](https://mlsanity.com/paper/2210.10127/) (2022)
- [Guided Cooperation in Hierarchical Reinforcement Learning via Model-based Rollout](https://mlsanity.com/paper/2309.13508/) (2023)
- [Accelerated Sim-to-Real Deep Reinforcement Learning: Learning Collision Avoidance from Human Player](https://mlsanity.com/paper/2102.10711/) (2021)
- [Differentiable Robust LQR Layers](https://mlsanity.com/paper/2106.05535/) (2021)
- [Integrating Expert Guidance for Efficient Learning of Safe Overtaking in Autonomous Driving Using Deep Reinforcement Learning](https://mlsanity.com/paper/2308.09456/) (2023)

## Abstract (arXiv)

Model predictive control (MPC) is an effective method for controlling robotic systems, particularly autonomous aerial vehicles such as quadcopters. However, application of MPC can be computationally demanding, and typically requires estimating the state of the system, which can be challenging in complex, unstructured environments. Reinforcement learning can in principle forego the need for explicit state estimation and acquire a policy that directly maps sensor readings to actions, but is difficult to apply to unstable systems that are liable to fail catastrophically during training before an effective policy has been found. We propose to combine MPC with reinforcement learning in the framework of guided policy search, where MPC is used to generate data at training time, under full state observations provided by an instrumented training environment. This data is used to train a deep neural network policy, which is allowed to access only the raw observations from the vehicle's onboard sensors. After training, the neural network policy can successfully control the robot without knowledge of the full state, and at a fraction of the computational cost of MPC. We evaluate our method by learning obstacle avoidance policies for a simulated quadrotor, using simulated onboard sensors and no explicit state estimation at test time.