# Deep Drone Racing: Learning Agile Flight in Dynamic Environments

arXiv: https://arxiv.org/abs/1806.08548 · HTML: https://mlsanity.com/paper/1806.08548/

## Key facts

- Cited by 34 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 22 Jun 2018.

## Built on by (most cited first, 30 of 34)

- [Legged Locomotion in Challenging Terrains using Egocentric Vision](https://mlsanity.com/paper/2211.07638/) (2022)
- [Neural Volumetric Memory for Visual Locomotion Control](https://mlsanity.com/paper/2304.01201/) (2023)
- [Learning a State Representation and Navigation in Cluttered and Dynamic Environments](https://mlsanity.com/paper/2103.04351/) (2021)
- [OmniDrones: An Efficient and Flexible Platform for Reinforcement Learning in Drone Control](https://mlsanity.com/paper/2309.12825/) (2023)
- [Demonstrating Agile Flight from Pixels without State Estimation](https://mlsanity.com/paper/2406.12505/) (2024)
- [PACT: Perception-Action Causal Transformer for Autoregressive Robotics Pre-Training](https://mlsanity.com/paper/2209.11133/) (2022)
- [Optimizing Nitrogen Management with Deep Reinforcement Learning and Crop Simulations](https://mlsanity.com/paper/2204.10394/) (2022)
- [Tiny Robot Learning: Challenges and Directions for Machine Learning in Resource-Constrained Robots](https://mlsanity.com/paper/2205.05748/) (2022)
- [Learning Deep Sensorimotor Policies for Vision-based Autonomous Drone Racing](https://mlsanity.com/paper/2210.14985/) (2022)
- [Dream to Fly: Model-Based Reinforcement Learning for Vision-Based Drone Flight](https://mlsanity.com/paper/2501.14377/) (2025)
- [Optimizing Crop Management with Reinforcement Learning and Imitation Learning](https://mlsanity.com/paper/2209.09991/) (2022)
- [The New Agronomists: Language Models are Experts in Crop Management](https://mlsanity.com/paper/2403.19839/) (2024)
- [Vision-based Learning for Drones: A Survey](https://mlsanity.com/paper/2312.05019/) (2023)
- [From explanation to synthesis: Compositional program induction for learning from demonstration](https://mlsanity.com/paper/1902.10657/) (2019)
- [HabiCrowd: A High Performance Simulator for Crowd-Aware Visual Navigation](https://mlsanity.com/paper/2306.11377/) (2023)
- [Invariance Through Latent Alignment](https://mlsanity.com/paper/2112.08526/) (2021)
- [Robust Navigation for Racing Drones based on Imitation Learning and Modularization](https://mlsanity.com/paper/2105.12923/) (2021)
- [One-shot Visual Imitation via Attributed Waypoints and Demonstration Augmentation](https://mlsanity.com/paper/2302.04856/) (2023)
- [BatMobility: Towards Flying Without Seeing for Autonomous Drones](https://mlsanity.com/paper/2307.11518/) (2023)
- [SPEAR: A Simulator for Photorealistic Embodied AI Research](https://mlsanity.com/paper/2607.06701/) (2026)
- [Self-Supervised Steering Angle Prediction for Vehicle Control Using Visual Odometry](https://mlsanity.com/paper/2103.11204/) (2021)
- [Machine Learning Subsystem for Autonomous Collision Avoidance on a small UAS with Embedded GPU](https://mlsanity.com/paper/2112.01688/) (2021)
- [RARA: Zero-shot Sim2Real Visual Navigation with Following Foreground Cues](https://mlsanity.com/paper/2201.02798/) (2022)
- [Approximate Imitation Learning for Event-based Quadrotor Flight in Cluttered Environments](https://mlsanity.com/paper/2603.07578/) (2026)
- [MotionScape: A Motion-Stratified UAV Video Benchmark for World Modeling and Future Video Generation](https://mlsanity.com/paper/2604.07991/) (2026)
- [Robustifying Reinforcement Learning Policies with L1 Adaptive Control](https://mlsanity.com/paper/2106.02249/) (2021)
- [DAL: Feature Learning from Overt Speech to Decode Imagined Speech-based EEG Signals with Convolutional Autoencoder](https://mlsanity.com/paper/2107.07064/) (2021)
- [ABPT: Amended Backpropagation through Time with Partially Differentiable Rewards](https://mlsanity.com/paper/2501.14513/) (2025)
- [SPIRAL: Self-Play Incremental Racing Algorithm for Learning in Multi-Drone Competitions](https://mlsanity.com/paper/2510.22568/) (2025)
- [Curriculum-Based Iterative Self-Play for Scalable Multi-Drone Racing](https://mlsanity.com/paper/2510.22570/) (2025)

## Abstract (arXiv)

Autonomous agile flight brings up fundamental challenges in robotics, such as coping with unreliable state estimation, reacting optimally to dynamically changing environments, and coupling perception and action in real time under severe resource constraints. In this paper, we consider these challenges in the context of autonomous, vision-based drone racing in dynamic environments. Our approach combines a convolutional neural network (CNN) with a state-of-the-art path-planning and control system. The CNN directly maps raw images into a robust representation in the form of a waypoint and desired speed. This information is then used by the planner to generate a short, minimum-jerk trajectory segment and corresponding motor commands to reach the desired goal. We demonstrate our method in autonomous agile flight scenarios, in which a vision-based quadrotor traverses drone-racing tracks with possibly moving gates. Our method does not require any explicit map of the environment and runs fully onboard. We extensively test the precision and robustness of the approach in simulation and in the physical world. We also evaluate our method against state-of-the-art navigation approaches and professional human drone pilots.