# Online Planning for Multi-UAV Pursuit-Evasion in Unknown Environments Using Deep Reinforcement Learning

arXiv: https://arxiv.org/abs/2409.15866 · HTML: https://mlsanity.com/paper/2409.15866/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Placed in Multi-agent reinforcement learning, one of 15 directions in Deep reinforcement learning methods.
- Builds on 8 of its 36 references that are in the map.
- First posted on arXiv 24 Sep 2024.

## Placed in

- Deep reinforcement learning methods / [Multi-agent reinforcement learning](https://mlsanity.com/deeprl/multi-agent-reinforcement-learning/)

## Builds on (most cited first, 8 of 8)

- [Domain Randomization for Transferring Deep Neural Networks from Simulation to the Real World](https://mlsanity.com/paper/1703.06907/) (2017)
- [The Surprising Effectiveness of PPO in Cooperative, Multi-Agent Games](https://mlsanity.com/paper/2103.01955/) (2021)
- [Dueling Network Architectures for Deep Reinforcement Learning](https://mlsanity.com/paper/1511.06581/) (2015)
- [Deep Reinforcement Learning for Swarm Systems](https://mlsanity.com/paper/1807.06613/) (2018)
- [OmniDrones: An Efficient and Flexible Platform for Reinforcement Learning in Drone Control](https://mlsanity.com/paper/2309.12825/) (2023)
- [Decentralized Multi-Agent Pursuit using Deep Reinforcement Learning](https://mlsanity.com/paper/2010.08193/) (2020)
- [Multi-robot Cooperative Pursuit via Potential Field-Enhanced Reinforcement Learning](https://mlsanity.com/paper/2203.04700/) (2022)
- [Cooperative Pursuit with Multi-Pursuer and One Faster Free-moving Evader](https://mlsanity.com/paper/2001.04731/) (2020)

## Built on by (most cited first, 4 of 4)

- [VolleyBots: A Testbed for Multi-Drone Volleyball Game Combining Motion Control and Strategic Play](https://mlsanity.com/paper/2502.01932/) (2025)
- [Learned Controllers for Agile Quadrotors in Pursuit-Evasion Games](https://mlsanity.com/paper/2506.02849/) (2025)
- [Agile Interception of a Flying Target using Competitive Reinforcement Learning](https://mlsanity.com/paper/2603.16279/) (2026)
- [Decentralized End-to-End Multi-AAV Pursuit Using Predictive Spatio-Temporal Observation via Deep Reinforcement Learning](https://mlsanity.com/paper/2603.24238/) (2026)

## Abstract (arXiv)

Multi-UAV pursuit-evasion, where pursuers aim to capture evaders, poses a key challenge for UAV swarm intelligence. Multi-agent reinforcement learning (MARL) has demonstrated potential in modeling cooperative behaviors, but most RL-based approaches remain constrained to simplified simulations with limited dynamics or fixed scenarios. Previous attempts to deploy RL policy to real-world pursuit-evasion are largely restricted to two-dimensional scenarios, such as ground vehicles or UAVs at fixed altitudes. In this paper, we address multi-UAV pursuit-evasion by considering UAV dynamics and physical constraints. We introduce an evader prediction-enhanced network to tackle partial observability in cooperative strategy learning. Additionally, we propose an adaptive environment generator within MARL training, enabling higher exploration efficiency and better policy generalization across diverse scenarios. Simulations show our method significantly outperforms all baselines in challenging scenarios, generalizing to unseen scenarios with a 100% capture rate. Finally, we derive a feasible policy via a two-stage reward refinement and deploy the policy on real quadrotors in a zero-shot manner. To our knowledge, this is the first work to derive and deploy an RL-based policy using collective thrust and body rates control commands for multi-UAV pursuit-evasion in unknown environments. The open-source code and videos are available at https://sites.google.com/view/pursuit-evasion-rl.