# Autonomous Drone Racing with Deep Reinforcement Learning

arXiv: https://arxiv.org/abs/2103.08624 · HTML: https://mlsanity.com/paper/2103.08624/

## Key facts

- Cited by 60 papers in the map (data through 14 Oct 2026).
- Builds on 18 of its 31 references that are in the map.
- First posted on arXiv 15 Mar 2021.

## Builds on (most cited first, 18 of 18)

- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [Learning Quadrupedal Locomotion over Challenging Terrain](https://mlsanity.com/paper/2010.11251/) (2020)
- [Deep Reinforcement Learning for Robotic Manipulation with Asynchronous Off-Policy Updates](https://mlsanity.com/paper/1610.00633/) (2016)
- [Survey of Deep Reinforcement Learning for Motion Planning of Autonomous Vehicles](https://mlsanity.com/paper/2001.11231/) (2020)
- [Control of a Quadrotor with Reinforcement Learning](https://mlsanity.com/paper/1707.05110/) (2017)
- [Flightmare: A Flexible Quadrotor Simulator](https://mlsanity.com/paper/2009.00563/) (2020)
- [Deep Drone Acrobatics](https://mlsanity.com/paper/2006.05768/) (2020)
- [Super-Human Performance in Gran Turismo Sport Using Deep Reinforcement Learning](https://mlsanity.com/paper/2008.07971/) (2020)
- [Optimization-Based Autonomous Racing of 1:43 Scale RC Cars](https://mlsanity.com/paper/1711.07300/) (2017)
- [Robust and Efficient Quadrotor Trajectory Generation for Fast Autonomous Flight](https://mlsanity.com/paper/1907.01531/) (2019)
- [End-to-End Race Driving with Deep Reinforcement Learning](https://mlsanity.com/paper/1807.02371/) (2018)
- [AlphaPilot: Autonomous Drone Racing](https://mlsanity.com/paper/2005.12813/) (2020)
- [FlightGoggles: A Modular Framework for Photorealistic Camera, Exteroceptive Sensor, and Dynamics Simulation](https://mlsanity.com/paper/1905.11377/) (2019)
- [Beauty and the Beast: Optimal Methods Meet Learning for Drone Racing](https://mlsanity.com/paper/1810.06224/) (2018)
- [AirSim Drone Racing Lab](https://mlsanity.com/paper/2003.05654/) (2020)
- [Minimum-time trajectory generation for quadrotors in constrained environments](https://mlsanity.com/paper/1706.06478/) (2017)
- [Multi-Fidelity Black-Box Optimization for Time-Optimal Quadrotor Maneuvers](https://mlsanity.com/paper/2006.02513/) (2020)
- [CPC: Complementary Progress Constraints for Time-Optimal Quadrotor Trajectories](https://mlsanity.com/paper/2007.06255/) (2020)

## Built on by (most cited first, 30 of 60)

- [Reaching the Limit in Autonomous Racing: Optimal Control versus Reinforcement Learning](https://mlsanity.com/paper/2310.10943/) (2023)
- [OmniDrones: An Efficient and Flexible Platform for Reinforcement Learning in Drone Control](https://mlsanity.com/paper/2309.12825/) (2023)
- [Learning to Fly in Seconds](https://mlsanity.com/paper/2311.13081/) (2023)
- [Bootstrapping Reinforcement Learning with Imitation for Vision-Based Agile Flight](https://mlsanity.com/paper/2403.12203/) (2024)
- [DexPBT: Scaling up Dexterous Manipulation for Hand-Arm Systems with Population Based Training](https://mlsanity.com/paper/2305.12127/) (2023)
- [Learning Perception-Aware Agile Flight in Cluttered Environments](https://mlsanity.com/paper/2210.01841/) (2022)
- [Training Efficient Controllers via Analytic Policy Gradient](https://mlsanity.com/paper/2209.13052/) (2022)
- [Learning a Single Near-hover Position Controller for Vastly Different Quadcopters](https://mlsanity.com/paper/2209.09232/) (2022)
- [Policy Search for Model Predictive Control with Application to Agile Drone Flight](https://mlsanity.com/paper/2112.03850/) (2021)
- [Optimizing Nitrogen Management with Deep Reinforcement Learning and Crop Simulations](https://mlsanity.com/paper/2204.10394/) (2022)
- [Multi-Task Reinforcement Learning for Quadrotors](https://mlsanity.com/paper/2412.12442/) (2024)
- [Learning Deep Sensorimotor Policies for Vision-based Autonomous Drone Racing](https://mlsanity.com/paper/2210.14985/) (2022)
- [Contrastive Learning for Enhancing Robust Scene Transfer in Vision-based Agile Flight](https://mlsanity.com/paper/2309.09865/) (2023)
- [Dream to Fly: Model-Based Reinforcement Learning for Vision-Based Drone Flight](https://mlsanity.com/paper/2501.14377/) (2025)
- [Measure gradients, not activations! Enhancing neuronal activity in deep reinforcement learning](https://mlsanity.com/paper/2505.24061/) (2025)
- [Position-Agnostic Autonomous Navigation in Vineyards with Deep Reinforcement Learning](https://mlsanity.com/paper/2206.14155/) (2022)
- [Vision-based Learning for Drones: A Survey](https://mlsanity.com/paper/2312.05019/) (2023)
- [What Matters in Learning A Zero-Shot Sim-to-Real RL Policy for Quadrotor Control? A Comprehensive Study](https://mlsanity.com/paper/2412.11764/) (2024)
- [Collaborative Target Search with a Visual Drone Swarm: An Adaptive Curriculum Embedded Multistage Reinforcement Learning Approach](https://mlsanity.com/paper/2204.12181/) (2022)
- [Learning Multi-Pursuit Evasion for Safe Targeted Navigation of Drones](https://mlsanity.com/paper/2304.03443/) (2023)
- [Contrastive Initial State Buffer for Reinforcement Learning](https://mlsanity.com/paper/2309.09752/) (2023)
- [Learning to Boost the Performance of Stable Nonlinear Systems](https://mlsanity.com/paper/2405.00871/) (2024)
- [Equivariant Reinforcement Learning for Quadrotor UAV](https://mlsanity.com/paper/2206.01233/) (2022)
- [Neural Moving Horizon Estimation for Robust Flight Control](https://mlsanity.com/paper/2206.10397/) (2022)
- [Probabilistic Verification of ReLU Neural Networks via Characteristic Functions](https://mlsanity.com/paper/2212.01544/) (2022)
- [Efficiently Learning Small Policies for Locomotion and Manipulation](https://mlsanity.com/paper/2210.00140/) (2022)
- [PIC4rl-gym: a ROS2 modular framework for Robots Autonomous Navigation with Deep Reinforcement Learning](https://mlsanity.com/paper/2211.10714/) (2022)
- [Optimality Principles in Spacecraft Neural Guidance and Control](https://mlsanity.com/paper/2305.13078/) (2023)
- [HyperPPO: A scalable method for finding small policies for robotic control](https://mlsanity.com/paper/2309.16663/) (2023)
- [Socialbots on Fire: Modeling Adversarial Behaviors of Socialbots via Multi-Agent Hierarchical Reinforcement Learning](https://mlsanity.com/paper/2110.10655/) (2021)

## Abstract (arXiv)

In many robotic tasks, such as autonomous drone racing, the goal is to travel through a set of waypoints as fast as possible. A key challenge for this task is planning the time-optimal trajectory, which is typically solved by assuming perfect knowledge of the waypoints to pass in advance. The resulting solution is either highly specialized for a single-track layout, or suboptimal due to simplifying assumptions about the platform dynamics. In this work, a new approach to near-time-optimal trajectory generation for quadrotors is presented. Leveraging deep reinforcement learning and relative gate observations, our approach can compute near-time-optimal trajectories and adapt the trajectory to environment changes. Our method exhibits computational advantages over approaches based on trajectory optimization for non-trivial track configurations. The proposed approach is evaluated on a set of race tracks in simulation and the real world, achieving speeds of up to 60 km/h with a physical quadrotor.