# Distributed Reinforcement Learning for Flexible and Efficient UAV Swarm Control

arXiv: https://arxiv.org/abs/2103.04666 · HTML: https://mlsanity.com/paper/2103.04666/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 13 of its 31 references that are in the map.
- First posted on arXiv 8 Mar 2021.

## Builds on (most cited first, 13 of 13)

- [Deep Reinforcement Learning with Double Q-learning](https://mlsanity.com/paper/1509.06461/) (2015)
- [On the Variance of the Adaptive Learning Rate and Beyond](https://mlsanity.com/paper/1908.03265/) (2019)
- [Learning to Communicate with Deep Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/1605.06676/) (2016)
- [A Survey and Critique of Multiagent Deep Reinforcement Learning](https://mlsanity.com/paper/1810.05587/) (2018)
- [A Survey of Learning in Multiagent Environments: Dealing with Non-Stationarity](https://mlsanity.com/paper/1707.09183/) (2017)
- [Convolutional Neural Networks at Constrained Time Cost](https://mlsanity.com/paper/1412.1710/) (2014)
- [Benchmark Analysis of Representative Deep Neural Network Architectures](https://mlsanity.com/paper/1810.00736/) (2018)
- [Multi-Agent Reinforcement Learning Based Resource Allocation for UAV Networks](https://mlsanity.com/paper/1810.10408/) (2018)
- [Unmanned Aerial Vehicles: A Survey on Civil Applications and Key Research Challenges](https://mlsanity.com/paper/1805.00881/) (2018)
- [Distributed Wildfire Surveillance with Autonomous Aircraft using Deep Reinforcement Learning](https://mlsanity.com/paper/1810.04244/) (2018)
- [Reinforcement Learning in Multiple-UAV Networks: Deployment and Movement Design](https://mlsanity.com/paper/1904.05242/) (2019)
- [Design Challenges of Multi-UAV Systems in Cyber-Physical Applications: A Comprehensive Survey, and Future Directions](https://mlsanity.com/paper/1810.09729/) (2018)
- [Reinforcement Learning for a Cellular Internet of UAVs: Protocol Design, Trajectory Control, and Resource Management](https://mlsanity.com/paper/1911.08771/) (2019)

## Built on by (most cited first, 3 of 3)

- [Sample and Communication-Efficient Decentralized Actor-Critic Algorithms with Finite-Time Analysis](https://mlsanity.com/paper/2109.03699/) (2021)
- [Neural Operators for PDE Backstepping Control of First-Order Hyperbolic PIDE with Recycle and Delay](https://mlsanity.com/paper/2307.11436/) (2023)
- [Distributed Machine Learning for UAV Swarms: Computing, Sensing, and Semantics](https://mlsanity.com/paper/2301.00912/) (2023)

## Abstract (arXiv)

Over the past few years, the use of swarms of Unmanned Aerial Vehicles (UAVs) in monitoring and remote area surveillance applications has become widespread thanks to the price reduction and the increased capabilities of drones. The drones in the swarm need to cooperatively explore an unknown area, in order to identify and monitor interesting targets, while minimizing their movements. In this work, we propose a distributed Reinforcement Learning (RL) approach that scales to larger swarms without modifications. The proposed framework relies on the possibility for the UAVs to exchange some information through a communication channel, in order to achieve context-awareness and implicitly coordinate the swarm's actions. Our experiments show that the proposed method can yield effective strategies, which are robust to communication channel impairments, and that can easily deal with non-uniform distributions of targets and obstacles. Moreover, when agents are trained in a specific scenario, they can adapt to a new one with minimal additional training. We also show that our approach achieves better performance compared to a computationally intensive look-ahead heuristic.