# Learning to Fly in Seconds

arXiv: https://arxiv.org/abs/2311.13081 · HTML: https://mlsanity.com/paper/2311.13081/

## Key facts

- Cited by 31 papers in the map (data through 14 Oct 2026).
- Builds on 29 of its 46 references that are in the map.
- First posted on arXiv 22 Nov 2023.

## Builds on (most cited first, 29 of 29)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [Continuous control with deep reinforcement learning](https://mlsanity.com/paper/1509.02971/) (2015)
- [Addressing Function Approximation Error in Actor-Critic Methods](https://mlsanity.com/paper/1802.09477/) (2018)
- [High-Dimensional Continuous Control Using Generalized Advantage Estimation](https://mlsanity.com/paper/1506.02438/) (2015)
- [Domain Randomization for Transferring Deep Neural Networks from Simulation to the Real World](https://mlsanity.com/paper/1703.06907/) (2017)
- [Solving Rubik's Cube with a Robot Hand](https://mlsanity.com/paper/1910.07113/) (2019)
- [Sim-to-Real Transfer of Robotic Control with Dynamics Randomization](https://mlsanity.com/paper/1710.06537/) (2017)
- [AirSim: High-Fidelity Visual and Physical Simulation for Autonomous Vehicles](https://mlsanity.com/paper/1705.05065/) (2017)
- [Asymmetric Actor Critic for Image-Based Robot Learning](https://mlsanity.com/paper/1710.06542/) (2017)
- [CAD2RL: Real Single-Image Flight without a Single Real Image](https://mlsanity.com/paper/1611.04201/) (2016)
- [Control of a Quadrotor with Reinforcement Learning](https://mlsanity.com/paper/1707.05110/) (2017)
- [Flightmare: A Flexible Quadrotor Simulator](https://mlsanity.com/paper/2009.00563/) (2020)
- [Learning to Fly -- a Gym Environment with PyBullet Physics for Reinforcement Learning of Multi-agent Quadcopter Control](https://mlsanity.com/paper/2103.02142/) (2021)
- [Deep Drone Acrobatics](https://mlsanity.com/paper/2006.05768/) (2020)
- [Autonomous Drone Racing with Deep Reinforcement Learning](https://mlsanity.com/paper/2103.08624/) (2021)
- [A Benchmark Comparison of Learned Control Policies for Agile Quadrotor Flight](https://mlsanity.com/paper/2202.10796/) (2022)
- [Sim-to-(Multi)-Real: Transfer of Low-Level Robust Control Policies to Multiple Quadrotors](https://mlsanity.com/paper/1903.04628/) (2019)
- [Low Level Control of a Quadrotor with Deep Model-Based Reinforcement Learning](https://mlsanity.com/paper/1901.03737/) (2019)
- [Model-Based Meta-Reinforcement Learning for Flight with Suspended Payloads](https://mlsanity.com/paper/2004.11345/) (2020)
- [Learning a Single Near-hover Position Controller for Vastly Different Quadcopters](https://mlsanity.com/paper/2209.09232/) (2022)
- [Learning to Fly via Deep Model-Based Reinforcement Learning](https://mlsanity.com/paper/2003.08876/) (2020)
- [Inclined Quadrotor Landing using Deep Reinforcement Learning](https://mlsanity.com/paper/2103.09043/) (2021)
- [Flight Controller Synthesis Via Deep Reinforcement Learning](https://mlsanity.com/paper/1909.06493/) (2019)
- [Autonomous Quadrotor Landing using Deep Reinforcement Learning](https://mlsanity.com/paper/1709.03339/) (2017)
- [Flying through a narrow gap using neural network: an end-to-end planning and control approach](https://mlsanity.com/paper/1903.09088/) (2019)
- [Using Simulation Optimization to Improve Zero-shot Policy Transfer of Quadrotors](https://mlsanity.com/paper/2201.01369/) (2022)
- [RLtools: A Fast, Portable Deep Reinforcement Learning Library for Continuous Control](https://mlsanity.com/paper/2306.03530/) (2023)
- [RotorTM: A Flexible Simulator for Aerial Transportation and Manipulation](https://mlsanity.com/paper/2205.05140/) (2022)

## Built on by (most cited first, 30 of 31)

- [Demonstrating Agile Flight from Pixels without State Estimation](https://mlsanity.com/paper/2406.12505/) (2024)
- [Multi-Task Reinforcement Learning for Quadrotors](https://mlsanity.com/paper/2412.12442/) (2024)
- [Learning Quadrotor Control From Visual Features Using Differentiable Simulation](https://mlsanity.com/paper/2410.15979/) (2024)
- [Dream to Fly: Model-Based Reinforcement Learning for Vision-Based Drone Flight](https://mlsanity.com/paper/2501.14377/) (2025)
- [What Matters in Learning A Zero-Shot Sim-to-Real RL Policy for Quadrotor Control? A Comprehensive Study](https://mlsanity.com/paper/2412.11764/) (2024)
- [RAPTOR: A Foundation Policy for Quadrotor Control](https://mlsanity.com/paper/2509.11481/) (2025)
- [One Net to Rule Them All: Domain Randomization in Quadcopter Racing Across Different Platforms](https://mlsanity.com/paper/2504.21586/) (2025)
- [Skill Transfer and Discovery for Sim-to-Real Learning: A Representation-Based Viewpoint](https://mlsanity.com/paper/2404.05051/) (2024)
- [Learning to Swim: Reinforcement Learning for 6-DOF Control of Thruster-driven Autonomous Underwater Vehicles](https://mlsanity.com/paper/2410.00120/) (2024)
- [Leveling the Playing Field: Carefully Comparing Classical and Learned Controllers for Quadrotor Trajectory Tracking](https://mlsanity.com/paper/2506.17832/) (2025)
- [ProxFly: Robust Control for Close Proximity Quadcopter Flight via Residual Reinforcement Learning](https://mlsanity.com/paper/2409.13193/) (2024)
- [Decentralized Aerial Manipulation of a Cable-Suspended Load using Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/2508.01522/) (2025)
- [RoVerFly: Robust and Versatile Implicit Hybrid Control of Quadrotor-Payload Systems](https://mlsanity.com/paper/2509.11149/) (2025)
- [Equivariant Neural Networks for General Linear Symmetries on Lie Algebras](https://mlsanity.com/paper/2510.22984/) (2025)
- [Equivariant Reinforcement Learning Frameworks for Quadrotor Low-Level Control](https://mlsanity.com/paper/2502.20500/) (2025)
- [Deep Reinforcement Learning-based Quadcopter Controller: A Practical Approach and Experiments](https://mlsanity.com/paper/2406.08815/) (2024)
- [Curriculum-based Sample Efficient Reinforcement Learning for Robust Stabilization of a Quadrotor](https://mlsanity.com/paper/2501.18490/) (2025)
- [Adaptive Surrogate Gradients for Sequential Reinforcement Learning in Spiking Neural Networks](https://mlsanity.com/paper/2510.24461/) (2025)
- [Efficient Knowledge Transfer for Jump-Starting Control Policy Learning of Multirotors through Physics-Aware Neural Architectures](https://mlsanity.com/paper/2602.15533/) (2026)
- [Crazyflow: An Accurate, GPU-Accelerated, Differentiable Drone Simulator in JAX](https://mlsanity.com/paper/2606.01478/) (2026)
- [Maintaining Plasticity in Reinforcement Learning: A Cost-Aware Framework for Aerial Robot Control in Non-stationary Environments](https://mlsanity.com/paper/2503.00282/) (2025)
- [Dynamics-Invariant Quadrotor Control using Scale-Aware Deep Reinforcement Learning](https://mlsanity.com/paper/2503.09622/) (2025)
- [Olfactory Inertial Odometry: Sensor Calibration and Drift Compensation](https://mlsanity.com/paper/2506.04539/) (2025)
- [MAVEN: A Meta-Reinforcement Learning Framework for Varying-Dynamics Expertise in Agile Quadrotor Maneuvers](https://mlsanity.com/paper/2603.10714/) (2026)
- [Aerial Inspection Behaviors via RL-based Quadrotor Control for Under-canopy Forest Environments](https://mlsanity.com/paper/2605.19202/) (2026)
- [MinNav: Minimalist Navigation Using Optical Flow For Active Tiny Aerial Robots](https://mlsanity.com/paper/2606.07813/) (2026)
- [Embodiment-conditioned Generalist Control for Multirotor Aerial Robots](https://mlsanity.com/paper/2606.10857/) (2026)
- [FlashNav: Training Deployable Robot Navigation Policies in Seconds](https://mlsanity.com/paper/2606.15846/) (2026)
- [Agile Fall Recovery for Quadrotors with Bidirectional Thrust via Reinforcement Learning](https://mlsanity.com/paper/2606.16513/) (2026)
- [A Single Diffusion-Policy Controller for Multi-Task Block Pushing with Zero-Shot Sim-to-Real Transfer](https://mlsanity.com/paper/2607.10892/) (2026)

## Abstract (arXiv)

Learning-based methods, particularly Reinforcement Learning (RL), hold great promise for streamlining deployment, enhancing performance, and achieving generalization in the control of autonomous multirotor aerial vehicles. Deep RL has been able to control complex systems with impressive fidelity and agility in simulation but the simulation-to-reality transfer often brings a hard-to-bridge reality gap. Moreover, RL is commonly plagued by prohibitively long training times. In this work, we propose a novel asymmetric actor-critic-based architecture coupled with a highly reliable RL-based training paradigm for end-to-end quadrotor control. We show how curriculum learning and a highly optimized simulator enhance sample complexity and lead to fast training times. To precisely discuss the challenges related to low-level/end-to-end multirotor control, we also introduce a taxonomy that classifies the existing levels of control abstractions as well as non-linearities and domain parameters. Our framework enables Simulation-to-Reality (Sim2Real) transfer for direct RPM control after only 18 seconds of training on a consumer-grade laptop as well as its deployment on microcontrollers to control a multirotor under real-time guarantees. Finally, our solution exhibits competitive performance in trajectory tracking, as demonstrated through various experimental comparisons with existing state-of-the-art control solutions using a real Crazyflie nano quadrotor. We open source the code including a very fast multirotor dynamics simulator that can simulate about 5 months of flight per second on a laptop GPU. The fast training times and deployment to a cheap, off-the-shelf quadrotor lower the barriers to entry and help democratize the research and development of these systems.