# Real-time Neural-MPC: Deep Learning Model Predictive Control for Quadrotors and Agile Robotic Platforms

arXiv: https://arxiv.org/abs/2203.07747 · HTML: https://mlsanity.com/paper/2203.07747/

## Key facts

- Cited by 41 papers in the map (data through 14 Oct 2026).
- Builds on 19 of its 37 references that are in the map.
- First posted on arXiv 15 Mar 2022.

## Builds on (most cited first, 19 of 19)

- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [Fourier Neural Operator for Parametric Partial Differential Equations](https://mlsanity.com/paper/2010.08895/) (2020)
- [Deep Reinforcement Learning in a Handful of Trials using Probabilistic Dynamics Models](https://mlsanity.com/paper/1805.12114/) (2018)
- [Learning Dexterous In-Hand Manipulation](https://mlsanity.com/paper/1808.00177/) (2018)
- [TensorFlow: Large-Scale Machine Learning on Heterogeneous Distributed Systems](https://mlsanity.com/paper/1603.04467/) (2016)
- [Learning agile and dynamic motor skills for legged robots](https://mlsanity.com/paper/1901.08652/) (2019)
- [Neural Lander: Stable Drone Landing Control using Learned Dynamics](https://mlsanity.com/paper/1811.08027/) (2018)
- [Deep Drone Acrobatics](https://mlsanity.com/paper/2006.05768/) (2020)
- [acados: a modular open-source framework for fast embedded optimal control](https://mlsanity.com/paper/1910.13753/) (2019)
- [Model-Predictive Policy Learning with Uncertainty Regularization for Driving in Dense Traffic](https://mlsanity.com/paper/1901.02705/) (2019)
- [Low Level Control of a Quadrotor with Deep Model-Based Reinforcement Learning](https://mlsanity.com/paper/1901.03737/) (2019)
- [NeuroBEM: Hybrid Aerodynamic Quadrotor Model](https://mlsanity.com/paper/2106.08015/) (2021)
- [Data-Driven MPC for Quadrotors](https://mlsanity.com/paper/2102.05773/) (2021)
- [Differential Flatness of Quadrotor Dynamics Subject to Rotor Drag for Accurate Tracking of High-Speed Trajectories](https://mlsanity.com/paper/1712.02402/) (2017)
- [PAMPC: Perception-Aware Model Predictive Control for Quadrotors](https://mlsanity.com/paper/1804.04811/) (2018)
- [KNODE-MPC: A Knowledge-based Data-driven Predictive Control Framework for Aerial Robots](https://mlsanity.com/paper/2109.04821/) (2021)
- [Learning Quadrotor Dynamics Using Neural Network for Flight Control](https://mlsanity.com/paper/1610.05863/) (2016)
- [A Neural Network Architecture to Learn Explicit MPC Controllers from Data](https://mlsanity.com/paper/1911.10789/) (2019)
- [On Training and Evaluation of Neural Network Approaches for Model Predictive Control](https://mlsanity.com/paper/2005.04112/) (2020)

## Built on by (most cited first, 30 of 41)

- [Real-Time Execution of Action Chunking Flow Policies](https://mlsanity.com/paper/2506.07339/) (2025)
- [Learning for CasADi: Data-driven Models in Numerical Optimization](https://mlsanity.com/paper/2312.05873/) (2023)
- [A Large Recurrent Action Model: xLSTM enables Fast Inference for Robotics Tasks](https://mlsanity.com/paper/2410.22391/) (2024)
- [RAMP-Net: A Robust Adaptive MPC for Quadrotors via Physics-informed Neural Network](https://mlsanity.com/paper/2209.09025/) (2022)
- [NeuPAN: Direct Point Robot Navigation with End-to-End Model-based Learning](https://mlsanity.com/paper/2403.06828/) (2024)
- [Feedback Favors the Generalization of Neural ODEs](https://mlsanity.com/paper/2410.10253/) (2024)
- [A view on learning robust goal-conditioned value functions: Interplay between RL and MPC](https://mlsanity.com/paper/2502.06996/) (2025)
- [AC4MPC: Actor-Critic Reinforcement Learning for Nonlinear Model Predictive Control](https://mlsanity.com/paper/2406.03995/) (2024)
- [The Power of Input: Benchmarking Zero-Shot Sim-To-Real Transfer of Reinforcement Learning Control Policies for Quadrotor Control](https://mlsanity.com/paper/2410.07686/) (2024)
- [Zero-Order Optimization for Gaussian Process-based Model Predictive Control](https://mlsanity.com/paper/2211.15522/) (2022)
- [Neural Potential Field for Obstacle-Aware Local Motion Planning](https://mlsanity.com/paper/2310.16362/) (2023)
- [Learning Maximal Safe Sets Using Hypernetworks for MPC-based Local Trajectory Planning in Unknown Environments](https://mlsanity.com/paper/2410.20267/) (2024)
- [Deep Operator Neural Network Model Predictive Control](https://mlsanity.com/paper/2505.18008/) (2025)
- [Closed-Loop Action Chunks with Dynamic Corrections for Training-Free Diffusion Policy](https://mlsanity.com/paper/2603.01953/) (2026)
- [Learning the References of Online Model Predictive Control for Urban Self-Driving](https://mlsanity.com/paper/2308.15808/) (2023)
- [ASV Station Keeping under Wind Disturbances using Neural Network Simulation Error Minimization Model Predictive Control](https://mlsanity.com/paper/2310.07892/) (2023)
- [Neural Horizon Model Predictive Control -- Increasing Computational Efficiency with Neural Networks](https://mlsanity.com/paper/2408.09781/) (2024)
- [Context-Aware Deep Lagrangian Networks for Model Predictive Control](https://mlsanity.com/paper/2506.15249/) (2025)
- [Deep active learning for nonlinear system identification](https://mlsanity.com/paper/2302.12667/) (2023)
- [Reinforcement Learning Compensated Model Predictive Control for Off-road Driving on Unknown Deformable Terrain](https://mlsanity.com/paper/2408.09253/) (2024)
- [DiffOP: Reinforcement Learning of Optimization-Based Control Policies via Implicit Policy Gradients](https://mlsanity.com/paper/2411.07484/) (2024)
- [Dynamic Tube MPC: Learning Tube Dynamics with Massively Parallel Simulation for Robust Safety in Practice](https://mlsanity.com/paper/2411.15350/) (2024)
- [Model-Free Predictive Control: Introductory Algebraic Calculations, and a Comparison with HEOL and ANNs](https://mlsanity.com/paper/2502.00443/) (2025)
- [PI-WAN: A Physics-Informed Wind-Adaptive Network for Quadrotor Dynamics Prediction in Unknown Environments](https://mlsanity.com/paper/2507.00816/) (2025)
- [Reservoir Predictive Path Integral Control for Unknown Nonlinear Dynamics](https://mlsanity.com/paper/2509.03839/) (2025)
- [Multi-Abstractive Neural Controller: An Efficient Hierarchical Control Architecture for Interactive Driving](https://mlsanity.com/paper/2305.14797/) (2023)
- [The Artificial Neural Twin -- Process Optimization and Continual Learning in Distributed Process Chains](https://mlsanity.com/paper/2403.18343/) (2024)
- [Efficient model predictive control for nonlinear systems modelled by deep neural networks](https://mlsanity.com/paper/2405.10372/) (2024)
- [Deep Reinforcement Learning Behavioral Mode Switching Using Optimal Control Based on a Latent Space Objective](https://mlsanity.com/paper/2406.01178/) (2024)
- [Dropout MPC: An Ensemble Neural MPC Approach for Systems with Learned Dynamics](https://mlsanity.com/paper/2406.02497/) (2024)

## Abstract (arXiv)

Model Predictive Control (MPC) has become a popular framework in embedded control for high-performance autonomous systems. However, to achieve good control performance using MPC, an accurate dynamics model is key. To maintain real-time operation, the dynamics models used on embedded systems have been limited to simple first-principle models, which substantially limits their representative power. In contrast to such simple models, machine learning approaches, specifically neural networks, have been shown to accurately model even complex dynamic effects, but their large computational complexity hindered combination with fast real-time iteration loops. With this work, we present Real-time Neural MPC, a framework to efficiently integrate large, complex neural network architectures as dynamics models within a model-predictive control pipeline. Our experiments, performed in simulation and the real world onboard a highly agile quadrotor platform, demonstrate the capabilities of the described system to run learned models with, previously infeasible, large modeling capacity using gradient-based online optimization MPC. Compared to prior implementations of neural networks in online optimization MPC we can leverage models of over 4000 times larger parametric capacity in a 50Hz real-time window on an embedded platform. Further, we show the feasibility of our framework on real-world problems by reducing the positional tracking error by up to 82% when compared to state-of-the-art MPC approaches without neural network dynamics.