mlsanityData to 14 Oct 2026

Policy Search for Model Predictive Control with Application to Agile Drone Flight

arXiv 2112.03850 ↗First posted 7 Dec 2021cs.RO cs.AIcited by 20

Not on the map. Why?

Abstract (arXiv)

Policy Search and Model Predictive Control~(MPC) are two different paradigms for robot control: policy search has the strength of automatically learning complex policies using experienced data, while MPC can offer optimal control performance using models and trajectory optimization. An open research question is how to leverage and combine the advantages of both approaches. In this work, we provide an answer by using policy search for automatically choosing high-level decision variables for MPC, which leads to a novel policy-search-for-model-predictive-control framework. Specifically, we formulate the MPC as a parameterized controller, where the hard-to-optimize decision variables are represented as high-level policies. Such a formulation allows optimizing policies in a self-supervised fashion. We validate this framework by focusing on a challenging problem in agile drone flight: flying a quadrotor through fast-moving gates. Experiments show that our controller achieves robust and real-time control performance in both simulation and the real world. The proposed framework offers a new perspective for merging learning and control.

Key facts. Cited by 20 papers in the map (data through 14 Oct 2026). Builds on 29 of its 79 references that are in the map. First posted on arXiv 7 Dec 2021.

Builds on

29 of 79 references.

References · 29

  1. 2015Trust Region Policy Optimizationcited by 2,580
  2. 2016OpenAI Gymcited by 1,736
  3. 2015End-to-End Training of Deep Visuomotor Policiescited by 875
  4. 2017Deep Reinforcement Learning that Matterscited by 754
  5. 2020Learning Quadrupedal Locomotion over Challenging Terraincited by 604
  6. 2019Learning agile and dynamic motor skills for legged robotscited by 504
  7. 2018Differentiable MPC for End-to-end Planning and Controlcited by 189
  8. 2017Control of a Quadrotor with Reinforcement Learningcited by 92
  9. 2017Information Theoretic Model Predictive Control: Theory and Applications to Autonomous Drivingcited by 90
  10. 2020Flightmare: A Flexible Quadrotor Simulatorcited by 84
  11. 2019Deep Drone Racing: From Simulation to Reality with Domain Randomizationcited by 76
  12. 2020Deep Drone Acrobaticscited by 75
17 more
  1. 2017Towards Generalization and Simplicity in Continuous Controlcited by 68
  2. 2021Autonomous Drone Racing with Deep Reinforcement Learningcited by 60
  3. 2021Autonomous Overtaking in Gran Turismo Sport Using Curriculum Reinforcement Learningcited by 45
  4. 2015Learning Deep Control Policies for Autonomous Aerial Vehicles with MPC-Guided Policy Searchcited by 42
  5. 2019Robust and Efficient Quadrotor Trajectory Generation for Fast Autonomous Flightcited by 41
  6. 2020Enforcing robust control guarantees within neural network policiescited by 40
  7. 2018A survey on policy search algorithms for learning robot controllers in a handful of trialscited by 35
  8. 2020AlphaPilot: Autonomous Drone Racingcited by 30
  9. 2018PAMPC: Perception-Aware Model Predictive Control for Quadrotorscited by 28
  10. 2020RAPTOR: Robust and Perception-aware Trajectory Replanning for Quadrotor Fast Flightcited by 23
  11. 2021Time-Optimal Planning for Quadrotor Waypoint Flightcited by 23
  12. 2018Beauty and the Beast: Optimal Methods Meet Learning for Drone Racingcited by 21
  13. 2017Search-based Motion Planning for Aggressive Flight in SE(3)cited by 15
  14. 2019Learning How to Autonomously Race a Car: a Predictive Control Approachcited by 14
  15. 2020Learning High-Level Policies for Model Predictive Controlcited by 10
  16. 2020Model Predictive Control for Micro Aerial Vehicles: A Surveycited by 9
  17. 2018Vision-Based High Speed Driving with a Deep Dynamic Observercited by 7

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2022: 3; 2023: 5; 2024: 5; 2025: 4; 2026: 3.

20 papers.

Sort:

By area:

  1. 2025Multi-agent Embodied AI: Advances and Future Directionscited by 36
  2. 2023Learning to Warm-Start Fixed-Point Optimization Algorithmscited by 32
  3. 2022Learning Perception-Aware Agile Flight in Cluttered Environmentscited by 23
  4. 2024Imperative Learning: A Self-supervised Neuro-Symbolic Learning Framework for Robot Autonomycited by 16
  5. 2024Safe Reinforcement Learning on the Constraint Manifold: Theory and Applicationscited by 15
  6. 2024Learning Speed Adaptation for Flight in Cluttercited by 14
  7. 2023Vision-based Learning for Drones: A Surveycited by 11
  8. 2022Inferring Smooth Control: Monte Carlo Posterior Policy Iteration with Gaussian Processescited by 10
  9. 2023Deep Model Predictive Optimizationcited by 7
  10. 2023Learning-Aided Warmstart of Model Predictive Control in Uncertain Fast-Changing Trafficcited by 4
  11. 2025Bilevel Multi-Armed Bandit-Based Hierarchical Reinforcement Learning for Interaction-Aware Self-Driving at Unsignalized Intersectionscited by 4
  12. 2025Generative AI for Autonomous Driving: A Reviewcited by 3
8 more
  1. 2022Bi-Level Optimization Augmented with Conditional Variational Autoencoder for Autonomous Driving in Dense Trafficcited by 2
  2. 2023Learning the References of Online Model Predictive Control for Urban Self-Drivingcited by 2
  3. 2024Toward Scalable Multirobot Control: Fast Policy Learning in Distributed MPCcited by 2
  4. 2025Learning Agile Gate Traversal via Analytical Optimal Policy Gradientcited by 1
  5. 2024Agile Robotics: Optimal Control, Reinforcement Learning, and Differentiable Simulationcited by 0
  6. 2026Merging model-based control with multi-agent reinforcement learning for multi-agent cooperative teaming strategiescited by 0
  7. 2026HOLO-MPPI: Multi-Scenario Motion Planning via Hierarchical Policy Optimizationcited by 0
  8. 2026Distributed Secure Learning Control for Large-scale Multirobots under Stealthy Actuator Attackscited by 0