# Decentralized Control of Quadrotor Swarms with End-to-end Deep Reinforcement Learning

arXiv: https://arxiv.org/abs/2109.07735 · HTML: https://mlsanity.com/paper/2109.07735/

## Key facts

- Cited by 19 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 16 Sep 2021.

## Built on by (most cited first, 19 of 19)

- [DexPBT: Scaling up Dexterous Manipulation for Hand-Arm Systems with Population Based Training](https://mlsanity.com/paper/2305.12127/) (2023)
- [Proximal Policy Gradient Arborescence for Quality Diversity Reinforcement Learning](https://mlsanity.com/paper/2305.13795/) (2023)
- [Communication-Efficient Large-Scale Distributed Deep Learning: A Comprehensive Survey](https://mlsanity.com/paper/2404.06114/) (2024)
- [Collision Avoidance and Navigation for a Quadrotor Swarm Using End-to-end Deep Reinforcement Learning](https://mlsanity.com/paper/2309.13285/) (2023)
- [Robust Multi-Agent Reinforcement Learning by Mutual Information Regularization](https://mlsanity.com/paper/2310.09833/) (2023)
- [Locally Interdependent Multi-Agent MDP: Theoretical Framework for Decentralized Agents with Dynamic Dependencies](https://mlsanity.com/paper/2406.06823/) (2024)
- [QuadSwarm: A Modular Multi-Quadrotor Simulator for Deep Reinforcement Learning with Direct Thrust Control](https://mlsanity.com/paper/2306.09537/) (2023)
- [Efficiently Learning Small Policies for Locomotion and Manipulation](https://mlsanity.com/paper/2210.00140/) (2022)
- [HyperPPO: A scalable method for finding small policies for robotic control](https://mlsanity.com/paper/2309.16663/) (2023)
- [Decentralized Aerial Manipulation of a Cable-Suspended Load using Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/2508.01522/) (2025)
- [Myopically Verifiable Probabilistic Certificates for Safe Control and Learning](https://mlsanity.com/paper/2404.16883/) (2024)
- [Symmetries-enhanced Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/2501.01136/) (2025)
- [Empirical Study on Robustness and Resilience in Cooperative Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/2510.11824/) (2025)
- [A Scalable Reinforcement Learning Approach for Attack Allocation in Swarm to Swarm Engagement Problems](https://mlsanity.com/paper/2210.08319/) (2022)
- [FPGA-Based Neural Thrust Controller for UAVs](https://mlsanity.com/paper/2403.18703/) (2024)
- [PRIMER: Perception-Aware Robust Learning-based Multiagent Trajectory Planner](https://mlsanity.com/paper/2406.10060/) (2024)
- [Mitigating Relative Over-Generalization in Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/2411.11099/) (2024)
- [Safe and Generalizable Hierarchical Multi-Agent RL via Constraint Manifold Control](https://mlsanity.com/paper/2606.24010/) (2026)
- [Visual Swarm Navigation via Deep Reinforcement Learning and Evolutionary Hybrid Design](https://mlsanity.com/paper/2610.06400/) (2026)

## Abstract (arXiv)

We demonstrate the possibility of learning drone swarm controllers that are zero-shot transferable to real quadrotors via large-scale multi-agent end-to-end reinforcement learning. We train policies parameterized by neural networks that are capable of controlling individual drones in a swarm in a fully decentralized manner. Our policies, trained in simulated environments with realistic quadrotor physics, demonstrate advanced flocking behaviors, perform aggressive maneuvers in tight formations while avoiding collisions with each other, break and re-establish formations to avoid collisions with moving obstacles, and efficiently coordinate in pursuit-evasion tasks. We analyze, in simulation, how different model architectures and parameters of the training regime influence the final performance of neural swarms. We demonstrate the successful deployment of the model learned in simulation to highly resource-constrained physical quadrotors performing station keeping and goal swapping behaviors. Code and video demonstrations are available on the project website at https://sites.google.com/view/swarm-rl.