# Dashing for the Golden Snitch: Multi-Drone Time-Optimal Motion Planning with Multi-Agent Reinforcement Learning

arXiv: https://arxiv.org/abs/2409.16720 · HTML: https://mlsanity.com/paper/2409.16720/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Builds on 22 of its 33 references that are in the map.
- First posted on arXiv 25 Sep 2024.

## Builds on (most cited first, 22 of 22)

- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [High-Dimensional Continuous Control Using Generalized Advantage Estimation](https://mlsanity.com/paper/1506.02438/) (2015)
- [The Surprising Effectiveness of PPO in Cooperative, Multi-Agent Games](https://mlsanity.com/paper/2103.01955/) (2021)
- [Is Independent Learning All You Need in the StarCraft Multi-Agent Challenge?](https://mlsanity.com/paper/2011.09533/) (2020)
- [Reaching the Limit in Autonomous Racing: Optimal Control versus Reinforcement Learning](https://mlsanity.com/paper/2310.10943/) (2023)
- [Scaling Multi-Agent Reinforcement Learning with Selective Parameter Sharing](https://mlsanity.com/paper/2102.07475/) (2021)
- [Towards Optimally Decentralized Multi-Robot Collision Avoidance via Deep Reinforcement Learning](https://mlsanity.com/paper/1709.10082/) (2017)
- [Learning to Fly -- a Gym Environment with PyBullet Physics for Reinforcement Learning of Multi-agent Quadcopter Control](https://mlsanity.com/paper/2103.02142/) (2021)
- [Back to Newton's Laws: Learning Vision-based Agile Flight via Differentiable Physics](https://mlsanity.com/paper/2407.10648/) (2024)
- [Differential Flatness of Quadrotor Dynamics Subject to Rotor Drag for Accurate Tracking of High-Speed Trajectories](https://mlsanity.com/paper/1712.02402/) (2017)
- [Time-Optimal Planning for Quadrotor Waypoint Flight](https://mlsanity.com/paper/2108.04537/) (2021)
- [MADER: Trajectory Planner in Multi-Agent and Dynamic Environments](https://mlsanity.com/paper/2010.11061/) (2020)
- [Model Predictive Contouring Control for Time-Optimal Quadrotor Flight](https://mlsanity.com/paper/2108.13205/) (2021)
- [Geometrically Constrained Trajectory Optimization for Multicopters](https://mlsanity.com/paper/2103.00190/) (2021)
- [End-to-end Reinforcement Learning for Time-Optimal Quadcopter Flight](https://mlsanity.com/paper/2311.16948/) (2023)
- [Vision Transformers for End-to-End Vision-Based Quadrotor Obstacle Avoidance](https://mlsanity.com/paper/2405.10391/) (2024)
- [Time-Optimal Online Replanning for Agile Quadrotor Flight](https://mlsanity.com/paper/2203.09839/) (2022)
- [Time-Optimal Gate-Traversing Planner for Autonomous Drone Racing](https://mlsanity.com/paper/2309.06837/) (2023)
- [AMSwarm: An Alternating Minimization Approach for Safe Motion Planning of Quadrotor Swarms in Cluttered Environments](https://mlsanity.com/paper/2303.04856/) (2023)
- [Euclidean and non-Euclidean Trajectory Optimization Approaches for Quadrotor Racing](https://mlsanity.com/paper/2309.07262/) (2023)
- [Aggressive Trajectory Generation for A Swarm of Autonomous Racing Drones](https://mlsanity.com/paper/2303.00851/) (2023)
- [AMSwarmX: Safe Swarm Coordination in CompleX Environments via Implicit Non-Convex Decomposition of the Obstacle-Free Space](https://mlsanity.com/paper/2310.09195/) (2023)

## Built on by (most cited first, 6 of 6)

- [Agile Flight Emerges from Multi-Agent Competitive Racing](https://mlsanity.com/paper/2512.11781/) (2025)
- [Towards Task-Oriented Flying: Framework, Infrastructure, and Principles](https://mlsanity.com/paper/2504.15129/) (2025)
- [FLARE: Agile Flights for Quadrotor Cable-Suspended Payload System via Reinforcement Learning](https://mlsanity.com/paper/2508.09797/) (2025)
- [Agile Interception of a Flying Target using Competitive Reinforcement Learning](https://mlsanity.com/paper/2603.16279/) (2026)
- [MAVEN: A Meta-Reinforcement Learning Framework for Varying-Dynamics Expertise in Agile Quadrotor Maneuvers](https://mlsanity.com/paper/2603.10714/) (2026)
- [ASTER: Attitude-aware Suspended-payload Quadrotor Traversal via Efficient Reinforcement Learning](https://mlsanity.com/paper/2603.10715/) (2026)

## Abstract (arXiv)

Recent innovations in autonomous drones have facilitated time-optimal flight in single-drone configurations, and enhanced maneuverability in multi-drone systems by applying optimal control and learning-based methods. However, few studies have achieved time-optimal motion planning for multi-drone systems, particularly during highly agile maneuvers or in dynamic scenarios. This paper presents a decentralized policy network using multi-agent reinforcement learning for time-optimal multi-drone flight. To strike a balance between flight efficiency and collision avoidance, we introduce a soft collision-free mechanism inspired by optimization-based methods. By customizing PPO in a centralized training, decentralized execution (CTDE) fashion, we unlock higher efficiency and stability in training while ensuring lightweight implementation. Extensive simulations show that, despite slight performance trade-offs compared to single-drone systems, our multi-drone approach maintains near-time-optimal performance with a low collision rate. Real-world experiments validate our method, with two quadrotors using the same network as in simulation achieving a maximum speed of 13.65 m/s and a maximum body rate of 13.4 rad/s in a 5.5 m * 5.5 m * 2.0 m space across various tracks, relying entirely on onboard computation.