# Collision Avoidance and Navigation for a Quadrotor Swarm Using End-to-end Deep Reinforcement Learning

arXiv: https://arxiv.org/abs/2309.13285 · HTML: https://mlsanity.com/paper/2309.13285/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- Builds on 17 of its 37 references that are in the map.
- First posted on arXiv 23 Sep 2023.

## Builds on (most cited first, 17 of 17)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [Policy Distillation](https://mlsanity.com/paper/1511.06295/) (2015)
- [Prioritized Level Replay](https://mlsanity.com/paper/2010.03934/) (2020)
- [Decentralized Non-communicating Multiagent Collision Avoidance with Deep Reinforcement Learning](https://mlsanity.com/paper/1609.07845/) (2016)
- [Sample Factory: Egocentric 3D Control from Pixels at 100000 FPS with Asynchronous Reinforcement Learning](https://mlsanity.com/paper/2006.11751/) (2020)
- [A Benchmark Comparison of Learned Control Policies for Agile Quadrotor Flight](https://mlsanity.com/paper/2202.10796/) (2022)
- [Sim-to-(Multi)-Real: Transfer of Low-Level Robust Control Policies to Multiple Quadrotors](https://mlsanity.com/paper/1903.04628/) (2019)
- [GLAS: Global-to-Local Safe Autonomy Synthesis for Multi-Robot Motion Planning with End-to-End Learning](https://mlsanity.com/paper/2002.11807/) (2020)
- [MADER: Trajectory Planner in Multi-Agent and Dynamic Environments](https://mlsanity.com/paper/2010.11061/) (2020)
- [Decentralized Control of Quadrotor Swarms with End-to-end Deep Reinforcement Learning](https://mlsanity.com/paper/2109.07735/) (2021)
- [Online Trajectory Generation with Distributed Model Predictive Control for Multi-Robot Motion Planning](https://mlsanity.com/paper/1909.05150/) (2019)
- [QuadSwarm: A Modular Multi-Quadrotor Simulator for Deep Reinforcement Learning with Direct Thrust Control](https://mlsanity.com/paper/2306.09537/) (2023)
- [Robust MADER: Decentralized and Asynchronous Multiagent Trajectory Planner Robust to Communication Delay](https://mlsanity.com/paper/2209.13667/) (2022)
- [Learning Observation-Based Certifiable Safe Policy for Decentralized Multi-Robot Navigation](https://mlsanity.com/paper/2109.07760/) (2021)
- [Probabilistic Trajectory Planning for Static and Interaction-aware Dynamic Obstacle Avoidance](https://mlsanity.com/paper/2302.12873/) (2023)
- [Reconfigurable formations of quadrotors on Lissajous curves for surveillance applications](https://mlsanity.com/paper/1812.04904/) (2018)

## Built on by (most cited first, 8 of 8)

- [Multi-Task Reinforcement Learning for Quadrotors](https://mlsanity.com/paper/2412.12442/) (2024)
- [HyperPPO: A scalable method for finding small policies for robotic control](https://mlsanity.com/paper/2309.16663/) (2023)
- [Scalable Safe Multi-Agent Reinforcement Learning for Multi-Agent System](https://mlsanity.com/paper/2501.13727/) (2025)
- [SDHN: Skewness-Driven Hypergraph Networks for Enhanced Localized Multi-Robot Coordination](https://mlsanity.com/paper/2504.06684/) (2025)
- [AttentionSwarm: Reinforcement Learning with Attention Control Barier Function for Crazyflie Drones in Dynamic Environments](https://mlsanity.com/paper/2503.07376/) (2025)
- [RL-based Control of UAS Subject to Significant Disturbance](https://mlsanity.com/paper/2504.08114/) (2025)
- [LEARN: Learning End-to-End Aerial Resource-Constrained Multi-Robot Navigation](https://mlsanity.com/paper/2511.17765/) (2025)
- [Communication-Free Collective Navigation for a Swarm of UAVs via LiDAR-Based Deep Reinforcement Learning](https://mlsanity.com/paper/2601.13657/) (2026)

## Abstract (arXiv)

End-to-end deep reinforcement learning (DRL) for quadrotor control promises many benefits -- easy deployment, task generalization and real-time execution capability. Prior end-to-end DRL-based methods have showcased the ability to deploy learned controllers onto single quadrotors or quadrotor teams maneuvering in simple, obstacle-free environments. However, the addition of obstacles increases the number of possible interactions exponentially, thereby increasing the difficulty of training RL policies. In this work, we propose an end-to-end DRL approach to control quadrotor swarms in environments with obstacles. We provide our agents a curriculum and a replay buffer of the clipped collision episodes to improve performance in obstacle-rich environments. We implement an attention mechanism to attend to the neighbor robots and obstacle interactions - the first successful demonstration of this mechanism on policies for swarm behavior deployed on severely compute-constrained hardware. Our work is the first work that demonstrates the possibility of learning neighbor-avoiding and obstacle-avoiding control policies trained with end-to-end DRL that transfers zero-shot to real quadrotors. Our approach scales to 32 robots with 80% obstacle density in simulation and 8 robots with 20% obstacle density in physical deployment. Video demonstrations are available on the project website at: https://sites.google.com/view/obst-avoid-swarm-rl.