# Agile Flight Emerges from Multi-Agent Competitive Racing

arXiv: https://arxiv.org/abs/2512.11781 · HTML: https://mlsanity.com/paper/2512.11781/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 23 of its 31 references that are in the map.
- First posted on arXiv 12 Dec 2025.

## Builds on (most cited first, 23 of 23)

- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [The Surprising Effectiveness of PPO in Cooperative, Multi-Agent Games](https://mlsanity.com/paper/2103.01955/) (2021)
- [Orbit: A Unified Simulation Framework for Interactive Robot Learning Environments](https://mlsanity.com/paper/2301.04195/) (2023)
- [Human-level performance in first-person multiplayer games with population-based deep reinforcement learning](https://mlsanity.com/paper/1807.01281/) (2018)
- [Reaching the Limit in Autonomous Racing: Optimal Control versus Reinforcement Learning](https://mlsanity.com/paper/2310.10943/) (2023)
- [Robust Autonomy Emerges from Self-Play](https://mlsanity.com/paper/2502.03349/) (2025)
- [Emergent Coordination Through Competition](https://mlsanity.com/paper/1902.07151/) (2019)
- [Autonomous Drone Racing: A Survey](https://mlsanity.com/paper/2301.01755/) (2023)
- [A learning agent that acquires social norms from public sanctions in decentralized multi-agent settings](https://mlsanity.com/paper/2106.09012/) (2021)
- [Time-Optimal Planning for Quadrotor Waypoint Flight](https://mlsanity.com/paper/2108.04537/) (2021)
- [A Real-Time Game Theoretic Planner for Autonomous Two-Player Drone Racing](https://mlsanity.com/paper/1801.02302/) (2018)
- [Model Predictive Contouring Control for Time-Optimal Quadrotor Flight](https://mlsanity.com/paper/2108.13205/) (2021)
- [End-to-end Reinforcement Learning for Time-Optimal Quadcopter Flight](https://mlsanity.com/paper/2311.16948/) (2023)
- [A Learning-based Quadcopter Controller with Extreme Adaptation](https://mlsanity.com/paper/2409.12949/) (2024)
- [End-to-end Neural Network Based Quadcopter control](https://mlsanity.com/paper/2304.13460/) (2023)
- [Time-Optimal Online Replanning for Agile Quadrotor Flight](https://mlsanity.com/paper/2203.09839/) (2022)
- [One Net to Rule Them All: Domain Randomization in Quadcopter Racing Across Different Platforms](https://mlsanity.com/paper/2504.21586/) (2025)
- [Dashing for the Golden Snitch: Multi-Drone Time-Optimal Motion Planning with Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/2409.16720/) (2024)
- [Leveling the Playing Field: Carefully Comparing Classical and Learned Controllers for Quadrotor Trajectory Tracking](https://mlsanity.com/paper/2506.17832/) (2025)
- [Adversarial Domain Randomization](https://mlsanity.com/paper/1812.00491/) (2018)
- [A Champion-level Vision-based Reinforcement Learning Agent for Competitive Racing in Gran Turismo 7](https://mlsanity.com/paper/2504.09021/) (2025)
- [Gate-Aware Online Planning for Two-Player Autonomous Drone Racing](https://mlsanity.com/paper/2402.18021/) (2024)
- [Non-Normalized Solutions of Generalized Nash Equilibrium in Autonomous Racing](https://mlsanity.com/paper/2503.12002/) (2025)

## Built on by (most cited first, 4 of 4)

- [Superhuman Safe and Agile Racing through Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/2605.22748/) (2026)
- [AI Coaching for Accelerating Human Skill Development with Reinforcement Learning](https://mlsanity.com/paper/2606.25337/) (2026)
- [Learning Robot Policies from Sparse Success Signals via STL-Guided Stein Variational Policy Gradient](https://mlsanity.com/paper/2609.31606/) (2026)
- [Battery-Aware Reinforcement Learning for Aggressive Quadrotor Flight](https://mlsanity.com/paper/2609.37316/) (2026)

## Abstract (arXiv)

Through multi-agent competition and the sparse high-level objective of winning a race, we find that both agile flight (e.g., high-speed motion pushing the platform to its physical limits) and strategy (e.g., overtaking or blocking) emerge from agents trained with reinforcement learning. We provide evidence in both simulation and the real world that this approach outperforms the common paradigm of training agents in isolation with rewards that prescribe behavior, e.g., progress on the raceline, in particular when the complexity of the environment increases, e.g., in the presence of obstacles. Moreover, we find that multi-agent competition yields policies that transfer more reliably to the real world than policies trained with a single-agent progress-based reward, despite the two methods using the same simulation environment, randomization strategy, and hardware. In addition to improved sim-to-real transfer, the multi-agent policies also exhibit some degree of generalization to opponents unseen at training time. Overall, our work, following in the tradition of multi-agent competitive game-play in digital domains, shows that sparse task-level rewards are sufficient for training agents capable of advanced low-level control in the physical world. Code: https://github.com/Jirl-upenn/AgileFlight_MultiAgent