# Low Level Control of a Quadrotor with Deep Model-Based Reinforcement Learning

arXiv: https://arxiv.org/abs/1901.03737 · HTML: https://mlsanity.com/paper/1901.03737/

## Key facts

- Cited by 44 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 11 Jan 2019.

## Built on by (most cited first, 30 of 44)

- [Reaching the Limit in Autonomous Racing: Optimal Control versus Reinforcement Learning](https://mlsanity.com/paper/2310.10943/) (2023)
- [Investigating Compounding Prediction Errors in Learned Dynamics Models](https://mlsanity.com/paper/2203.09637/) (2022)
- [MBRL-Lib: A Modular Library for Model-based Reinforcement Learning](https://mlsanity.com/paper/2104.10159/) (2021)
- [Real-time Neural-MPC: Deep Learning Model Predictive Control for Quadrotors and Agile Robotic Platforms](https://mlsanity.com/paper/2203.07747/) (2022)
- [OmniDrones: An Efficient and Flexible Platform for Reinforcement Learning in Drone Control](https://mlsanity.com/paper/2309.12825/) (2023)
- [Learning to Fly in Seconds](https://mlsanity.com/paper/2311.13081/) (2023)
- [Demonstrating Agile Flight from Pixels without State Estimation](https://mlsanity.com/paper/2406.12505/) (2024)
- [KNODE-MPC: A Knowledge-based Data-driven Predictive Control Framework for Aerial Robots](https://mlsanity.com/paper/2109.04821/) (2021)
- [The Challenges of Exploration for Offline Reinforcement Learning](https://mlsanity.com/paper/2201.11861/) (2022)
- [Synthesis of Model Predictive Control and Reinforcement Learning: Survey and Classification](https://mlsanity.com/paper/2502.02133/) (2025)
- [Learning Dynamics Models for Model Predictive Agents](https://mlsanity.com/paper/2109.14311/) (2021)
- [Learning a Single Near-hover Position Controller for Vastly Different Quadcopters](https://mlsanity.com/paper/2209.09232/) (2022)
- [Multi-Task Reinforcement Learning for Quadrotors](https://mlsanity.com/paper/2412.12442/) (2024)
- [DHRL: A Graph-Based Approach for Long-Horizon and Sparse Hierarchical Reinforcement Learning](https://mlsanity.com/paper/2210.05150/) (2022)
- [Online Dynamics Learning for Predictive Control with an Application to Aerial Robots](https://mlsanity.com/paper/2207.09344/) (2022)
- [Dream to Fly: Model-Based Reinforcement Learning for Vision-Based Drone Flight](https://mlsanity.com/paper/2501.14377/) (2025)
- [Real-Time Reinforcement Learning for Vision-Based Robotics Utilizing Local and Remote Computers](https://mlsanity.com/paper/2210.02317/) (2022)
- [Reinforcement Learning for Safe Robot Control using Control Lyapunov Barrier Functions](https://mlsanity.com/paper/2305.09793/) (2023)
- [Physics Embedded Neural Network Vehicle Model and Applications in Risk-Aware Autonomous Driving Using Latent Features](https://mlsanity.com/paper/2207.07920/) (2022)
- [NanoFlowNet: Real-time Dense Optical Flow on a Nano Quadcopter](https://mlsanity.com/paper/2209.06918/) (2022)
- [SkyJEPA: Learning Long-Horizon World Models for Zero-Shot Sim-to-Real Control of Quadrotors](https://mlsanity.com/paper/2606.23444/) (2026)
- [ModelLight: Model-Based Meta-Reinforcement Learning for Traffic Signal Control](https://mlsanity.com/paper/2111.08067/) (2021)
- [Inclined Quadrotor Landing using Deep Reinforcement Learning](https://mlsanity.com/paper/2103.09043/) (2021)
- [Bridging Active Exploration and Uncertainty-Aware Deployment Using Probabilistic Ensemble Neural Network Dynamics](https://mlsanity.com/paper/2305.12240/) (2023)
- [Data-Efficient Deep Reinforcement Learning for Attitude Control of Fixed-Wing UAVs: Field Experiments](https://mlsanity.com/paper/2111.04153/) (2021)
- [Using Simulation Optimization to Improve Zero-shot Policy Transfer of Quadrotors](https://mlsanity.com/paper/2201.01369/) (2022)
- [HyperPPO: A scalable method for finding small policies for robotic control](https://mlsanity.com/paper/2309.16663/) (2023)
- [Learning on Abstract Domains: A New Approach for Verifiable Guarantee in Reinforcement Learning](https://mlsanity.com/paper/2106.06931/) (2021)
- [Reinforcement Learning with Formal Performance Metrics for Quadcopter Attitude Control under Non-nominal Contexts](https://mlsanity.com/paper/2107.12942/) (2021)
- [LiRA: Light-Robust Adversary for Model-based Reinforcement Learning in Real World](https://mlsanity.com/paper/2409.19617/) (2024)

## Abstract (arXiv)

Designing effective low-level robot controllers often entail platform-specific implementations that require manual heuristic parameter tuning, significant system knowledge, or long design times. With the rising number of robotic and mechatronic systems deployed across areas ranging from industrial automation to intelligent toys, the need for a general approach to generating low-level controllers is increasing. To address the challenge of rapidly generating low-level controllers, we argue for using model-based reinforcement learning (MBRL) trained on relatively small amounts of automatically generated (i.e., without system simulation) data. In this paper, we explore the capabilities of MBRL on a Crazyflie centimeter-scale quadrotor with rapid dynamics to predict and control at <50Hz. To our knowledge, this is the first use of MBRL for controlled hover of a quadrotor using only on-board sensors, direct motor input signals, and no initial dynamics knowledge. Our controller leverages rapid simulation of a neural network forward dynamics model on a GPU-enabled base station, which then transmits the best current action to the quadrotor firmware via radio. In our experiments, the quadrotor achieved hovering capability of up to 6 seconds with 3 minutes of experimental training data.