# End-to-end Reinforcement Learning for Time-Optimal Quadcopter Flight

arXiv: https://arxiv.org/abs/2311.16948 · HTML: https://mlsanity.com/paper/2311.16948/

## Key facts

- Cited by 17 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 28 Nov 2023.

## Built on by (most cited first, 17 of 17)

- [Demonstrating Agile Flight from Pixels without State Estimation](https://mlsanity.com/paper/2406.12505/) (2024)
- [See, Point, Fly: A Learning-Free VLM Framework for Universal Unmanned Aerial Navigation](https://mlsanity.com/paper/2509.22653/) (2025)
- [Dream to Fly: Model-Based Reinforcement Learning for Vision-Based Drone Flight](https://mlsanity.com/paper/2501.14377/) (2025)
- [RAPTOR: A Foundation Policy for Quadrotor Control](https://mlsanity.com/paper/2509.11481/) (2025)
- [SkyDreamer: Interpretable End-to-End Vision-Based Drone Racing with Model-Based Reinforcement Learning](https://mlsanity.com/paper/2510.14783/) (2025)
- [One Net to Rule Them All: Domain Randomization in Quadcopter Racing Across Different Platforms](https://mlsanity.com/paper/2504.21586/) (2025)
- [Dashing for the Golden Snitch: Multi-Drone Time-Optimal Motion Planning with Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/2409.16720/) (2024)
- [ProxFly: Robust Control for Close Proximity Quadcopter Flight via Residual Reinforcement Learning](https://mlsanity.com/paper/2409.13193/) (2024)
- [Neuromorphic Attitude Estimation and Control](https://mlsanity.com/paper/2411.13945/) (2024)
- [Agile Flight Emerges from Multi-Agent Competitive Racing](https://mlsanity.com/paper/2512.11781/) (2025)
- [DiffAero: A GPU-Accelerated Differentiable Simulation Framework for Efficient Quadrotor Policy Learning](https://mlsanity.com/paper/2509.10247/) (2025)
- [Mastering Diverse, Unknown, and Cluttered Tracks for Robust Vision-Based Drone Racing](https://mlsanity.com/paper/2512.09571/) (2025)
- [Learning human-to-robot handovers through 3D scene reconstruction](https://mlsanity.com/paper/2507.08726/) (2025)
- [Comparing Behavioural Cloning and Reinforcement Learning for Spacecraft Guidance and Control Networks](https://mlsanity.com/paper/2507.19535/) (2025)
- [MAVEN: A Meta-Reinforcement Learning Framework for Varying-Dynamics Expertise in Agile Quadrotor Maneuvers](https://mlsanity.com/paper/2603.10714/) (2026)
- [Vision-Guided Outdoor Flight and Obstacle Evasion via Reinforcement Learning](https://mlsanity.com/paper/2605.24449/) (2026)
- [Physics-Aware End-to-End Deep Reinforcement Learning for Quadcopter Control with Actuator Dynamics](https://mlsanity.com/paper/2607.25985/) (2026)

## Abstract (arXiv)

Aggressive time-optimal control of quadcopters poses a significant challenge in the field of robotics. The state-of-the-art approach leverages reinforcement learning (RL) to train optimal neural policies. However, a critical hurdle is the sim-to-real gap, often addressed by employing a robust inner loop controller -an abstraction that, in theory, constrains the optimality of the trained controller, necessitating margins to counter potential disturbances. In contrast, our novel approach introduces high-speed quadcopter control using end-to-end RL (E2E) that gives direct motor commands. To bridge the reality gap, we incorporate a learned residual model and an adaptive method that can compensate for modeling errors in thrust and moments. We compare our E2E approach against a state-of-the-art network that commands thrust and body rates to an INDI inner loop controller, both in simulated and real-world flight. E2E showcases a significant 1.39-second advantage in simulation and a 0.17-second edge in real-world testing, highlighting end-to-end reinforcement learning's potential. The performance drop observed from simulation to reality shows potential for further improvement, including refining strategies to address the reality gap or exploring offline reinforcement learning with real flight data.