# Control of a Quadrotor with Reinforcement Learning

arXiv: https://arxiv.org/abs/1707.05110 · HTML: https://mlsanity.com/paper/1707.05110/

## Key facts

- Cited by 92 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 17 Jul 2017.

## Built on by (most cited first, 30 of 92)

- [Deep Reinforcement Learning for Robotics: A Survey of Real-World Successes](https://mlsanity.com/paper/2408.03539/) (2024)
- [Reaching the Limit in Autonomous Racing: Optimal Control versus Reinforcement Learning](https://mlsanity.com/paper/2310.10943/) (2023)
- [Autonomous Drone Racing with Deep Reinforcement Learning](https://mlsanity.com/paper/2103.08624/) (2021)
- [Holo-Dex: Teaching Dexterity with Immersive Mixed Reality](https://mlsanity.com/paper/2210.06463/) (2022)
- [Not Only Rewards But Also Constraints: Applications on Legged Robot Locomotion](https://mlsanity.com/paper/2308.12517/) (2023)
- [OmniDrones: An Efficient and Flexible Platform for Reinforcement Learning in Drone Control](https://mlsanity.com/paper/2309.12825/) (2023)
- [Learning to Fly in Seconds](https://mlsanity.com/paper/2311.13081/) (2023)
- [Physics-informed Dyna-Style Model-Based Deep Reinforcement Learning for Dynamic Control](https://mlsanity.com/paper/2108.00128/) (2021)
- [Robust High-speed Running for Quadruped Robots via Deep Reinforcement Learning](https://mlsanity.com/paper/2103.06484/) (2021)
- [Learning a Single Near-hover Position Controller for Vastly Different Quadcopters](https://mlsanity.com/paper/2209.09232/) (2022)
- [Policy Search for Model Predictive Control with Application to Agile Drone Flight](https://mlsanity.com/paper/2112.03850/) (2021)
- [Adaptive Horizon Actor-Critic for Policy Learning in Contact-Rich Differentiable Simulation](https://mlsanity.com/paper/2405.17784/) (2024)
- [Multi-Task Reinforcement Learning for Quadrotors](https://mlsanity.com/paper/2412.12442/) (2024)
- [DHRL: A Graph-Based Approach for Long-Horizon and Sparse Hierarchical Reinforcement Learning](https://mlsanity.com/paper/2210.05150/) (2022)
- [Learning Quadrotor Control From Visual Features Using Differentiable Simulation](https://mlsanity.com/paper/2410.15979/) (2024)
- [Backward Reachability Analysis of Neural Feedback Loops: Techniques for Linear and Nonlinear Systems](https://mlsanity.com/paper/2209.14076/) (2022)
- [See, Point, Fly: A Learning-Free VLM Framework for Universal Unmanned Aerial Navigation](https://mlsanity.com/paper/2509.22653/) (2025)
- [DATT: Deep Adaptive Trajectory Tracking for Quadrotor Control](https://mlsanity.com/paper/2310.09053/) (2023)
- [Learning Deep Sensorimotor Policies for Vision-based Autonomous Drone Racing](https://mlsanity.com/paper/2210.14985/) (2022)
- [What Matters in Learning A Zero-Shot Sim-to-Real RL Policy for Quadrotor Control? A Comprehensive Study](https://mlsanity.com/paper/2412.11764/) (2024)
- [Boosting Reinforcement Learning with Strongly Delayed Feedback Through Auxiliary Short Delays](https://mlsanity.com/paper/2402.03141/) (2024)
- [Gradient-based Regularization for Action Smoothness in Robotic Control with Reinforcement Learning](https://mlsanity.com/paper/2407.04315/) (2024)
- [Variational Delayed Policy Optimization](https://mlsanity.com/paper/2405.14226/) (2024)
- [Inclined Quadrotor Landing using Deep Reinforcement Learning](https://mlsanity.com/paper/2103.09043/) (2021)
- [Deep Model Predictive Optimization](https://mlsanity.com/paper/2310.04590/) (2023)
- [DEER: A Delay-Resilient Framework for Reinforcement Learning with Variable Delays](https://mlsanity.com/paper/2406.03102/) (2024)
- [Equivariant Reinforcement Learning for Quadrotor UAV](https://mlsanity.com/paper/2206.01233/) (2022)
- [Environment as Policy: Learning to Race in Unseen Tracks](https://mlsanity.com/paper/2410.22308/) (2024)
- [Self-Improving Semantic Perception for Indoor Localisation](https://mlsanity.com/paper/2105.01595/) (2021)
- [Bridging Model-based Safety and Model-free Reinforcement Learning through System Identification of Low Dimensional Linear Models](https://mlsanity.com/paper/2205.05787/) (2022)

## Abstract (arXiv)

In this paper, we present a method to control a quadrotor with a neural network trained using reinforcement learning techniques. With reinforcement learning, a common network can be trained to directly map state to actuator command making any predefined control structure obsolete for training. Moreover, we present a new learning algorithm which differs from the existing ones in certain aspects. Our algorithm is conservative but stable for complicated tasks. We found that it is more applicable to controlling a quadrotor than existing algorithms. We demonstrate the performance of the trained policy both in simulation and with a real quadrotor. Experiments show that our policy network can react to step response relatively accurately. With the same policy, we also demonstrate that we can stabilize the quadrotor in the air even under very harsh initialization (manually throwing it upside-down in the air with an initial velocity of 5 m/s). Computation time of evaluating the policy is only 7 μs per time step which is two orders of magnitude less than common trajectory optimization algorithms with an approximated model.