# Collaborative Target Search with a Visual Drone Swarm: An Adaptive Curriculum Embedded Multistage Reinforcement Learning Approach arXiv: https://arxiv.org/abs/2204.12181 ยท HTML: https://mlsanity.com/paper/2204.12181/ ## Key facts - Cited by 9 papers in the map (data through 14 Oct 2026). - Builds on 26 of its 64 references that are in the map. - First posted on arXiv 26 Apr 2022. ## Builds on (most cited first, 26 of 26) - [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017) - [Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actor](https://mlsanity.com/paper/1801.01290/) (2018) - [Domain Randomization for Transferring Deep Neural Networks from Simulation to the Real World](https://mlsanity.com/paper/1703.06907/) (2017) - [Dota 2 with Large Scale Deep Reinforcement Learning](https://mlsanity.com/paper/1912.06680/) (2019) - [The Surprising Effectiveness of PPO in Cooperative, Multi-Agent Games](https://mlsanity.com/paper/2103.01955/) (2021) - [IMPALA: Scalable Distributed Deep-RL with Importance Weighted Actor-Learner Architectures](https://mlsanity.com/paper/1802.01561/) (2018) - [Deep Reinforcement Learning that Matters](https://mlsanity.com/paper/1709.06560/) (2017) - [Counterfactual Multi-Agent Policy Gradients](https://mlsanity.com/paper/1705.08926/) (2017) - [Solving Rubik's Cube with a Robot Hand](https://mlsanity.com/paper/1910.07113/) (2019) - [Learning Dexterous In-Hand Manipulation](https://mlsanity.com/paper/1808.00177/) (2018) - [Target-driven Visual Navigation in Indoor Scenes using Deep Reinforcement Learning](https://mlsanity.com/paper/1609.05143/) (2016) - [Unity: A General Platform for Intelligent Agents](https://mlsanity.com/paper/1809.02627/) (2018) - [Learning to Navigate in Complex Environments](https://mlsanity.com/paper/1611.03673/) (2016) - [Learning High-Speed Flight in the Wild](https://mlsanity.com/paper/2110.05113/) (2021) - [Stabilising Experience Replay for Deep Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/1702.08887/) (2017) - [PRIMAL: Pathfinding via Reinforcement and Imitation Multi-Agent Learning](https://mlsanity.com/paper/1809.03531/) (2018) - [PRIMAL2: Pathfinding via Reinforcement and Imitation Multi-Agent Learning -- Lifelong](https://mlsanity.com/paper/2010.08184/) (2020) - [Autonomous Drone Racing with Deep Reinforcement Learning](https://mlsanity.com/paper/2103.08624/) (2021) - [CM3: Cooperative Multi-goal Multi-stage Multi-agent Reinforcement Learning](https://mlsanity.com/paper/1809.05188/) (2018) - [Integrated Task Assignment and Path Planning for Capacitated Multi-Agent Pickup and Delivery](https://mlsanity.com/paper/2110.14891/) (2021) - [Towards Target-Driven Visual Navigation in Indoor Scenes via Generative Imitation Learning](https://mlsanity.com/paper/2009.14509/) (2020) - [Sniffy Bug: A Fully Autonomous Swarm of Gas-Seeking Nano Quadcopters in Cluttered Environments](https://mlsanity.com/paper/2107.05490/) (2021) - [Image-Goal Navigation in Complex Environments via Modular Learning](https://mlsanity.com/paper/2108.06105/) (2021) - [Flying Through a Narrow Gap Using End-to-end Deep Reinforcement Learning Augmented with Curriculum Learning and Sim2Real](https://mlsanity.com/paper/2108.12869/) (2021) - [Drone swarm patrolling with uneven coverage requirements](https://mlsanity.com/paper/2107.00362/) (2021) - [Multi-agent Time-based Decision-making for the Search and Action Problem](https://mlsanity.com/paper/1802.10147/) (2018) ## Built on by (most cited first, 9 of 9) - [Grounded Vision-Language Navigation for UAVs with Open-Vocabulary Goal Understanding](https://mlsanity.com/paper/2506.10756/) (2025) - [Vision-based Learning for Drones: A Survey](https://mlsanity.com/paper/2312.05019/) (2023) - [Learning Multi-Pursuit Evasion for Safe Targeted Navigation of Drones](https://mlsanity.com/paper/2304.03443/) (2023) - [Target Search and Navigation in Heterogeneous Robot Systems with Deep Reinforcement Learning](https://mlsanity.com/paper/2308.00331/) (2023) - [Human-In-The-Loop Machine Learning for Safe and Ethical Autonomous Vehicles: Principles, Challenges, and Opportunities](https://mlsanity.com/paper/2408.12548/) (2024) - [FM-Planner: Foundation Model Guided Path Planning for Autonomous Drone Navigation](https://mlsanity.com/paper/2505.20783/) (2025) - [Oracle-Guided Masked Contrastive Reinforcement Learning for Visuomotor Policies](https://mlsanity.com/paper/2510.05692/) (2025) - [GaussFly: Contrastive Reinforcement Learning for Visuomotor Policies in 3D Gaussian Fields](https://mlsanity.com/paper/2604.05062/) (2026) - [Curriculum-Based Adversarial Heterogeneous Agent Reinforcement Learning for Autonomous Quad-Copter Landing in Maritime Settings](https://mlsanity.com/paper/2609.12758/) (2026) ## Abstract (arXiv) Equipping drones with target search capabilities is highly desirable for applications in disaster rescue and smart warehouse delivery systems. Multiple intelligent drones that can collaborate with each other and maneuver among obstacles show more effectiveness in accomplishing tasks in a shorter amount of time. However, carrying out collaborative target search (CTS) without prior target information is extremely challenging, especially with a visual drone swarm. In this work, we propose a novel data-efficient deep reinforcement learning (DRL) approach called adaptive curriculum embedded multistage learning (ACEMSL) to address these challenges, mainly 3-D sparse reward space exploration with limited visual perception and collaborative behavior requirements. Specifically, we decompose the CTS task into several subtasks including individual obstacle avoidance, target search, and inter-agent collaboration, and progressively train the agents with multistage learning. Meanwhile, an adaptive embedded curriculum (AEC) is designed, where the task difficulty level (TDL) can be adaptively adjusted based on the success rate (SR) achieved in training. ACEMSL allows data-efficient training and individual-team reward allocation for the visual drone swarm. Furthermore, we deploy the trained model over a real visual drone swarm and perform CTS operations without fine-tuning. Extensive simulations and real-world flight tests validate the effectiveness and generalizability of ACEMSL. The project is available at https://github.com/NTU-UAVG/CTS-visual-drone-swarm.git.