# ARBoids: Adaptive Residual Reinforcement Learning With Boids Model for Cooperative Multi-USV Target Defense

arXiv: https://arxiv.org/abs/2502.18549 · HTML: https://mlsanity.com/paper/2502.18549/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in RL for robot navigation and UAV control, one of 15 directions in Deep reinforcement learning methods.
- Builds on 15 of its 41 references that are in the map.
- First posted on arXiv 25 Feb 2025.

## Placed in

- Deep reinforcement learning methods / [RL for robot navigation and UAV control](https://mlsanity.com/deeprl/rl-for-robot-navigation-and-uav-control/)

## Builds on (most cited first, 15 of 15)

- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actor](https://mlsanity.com/paper/1801.01290/) (2018)
- [Multi-Agent Actor-Critic for Mixed Cooperative-Competitive Environments](https://mlsanity.com/paper/1706.02275/) (2017)
- [Monotonic Value Function Factorisation for Deep Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/2003.08839/) (2020)
- [Residual Reinforcement Learning for Robot Control](https://mlsanity.com/paper/1812.03201/) (2018)
- [Deep Reinforcement Learning for Swarm Systems](https://mlsanity.com/paper/1807.06613/) (2018)
- [Residual Skill Policies: Learning an Adaptable Skill-based Action Space for Reinforcement Learning for Robotics](https://mlsanity.com/paper/2211.02231/) (2022)
- [Decentralized Multi-Agent Pursuit using Deep Reinforcement Learning](https://mlsanity.com/paper/2010.08193/) (2020)
- [Survey of Deep Learning for Autonomous Surface Vehicles in the Marine Environment](https://mlsanity.com/paper/2210.08487/) (2022)
- [Multi-robot Cooperative Pursuit via Potential Field-Enhanced Reinforcement Learning](https://mlsanity.com/paper/2203.04700/) (2022)
- [Cooperative Pursuit with Multi-Pursuer and One Faster Free-moving Evader](https://mlsanity.com/paper/2001.04731/) (2020)
- [Guarding a Target Set from a Single Attacker in the Euclidean Space](https://mlsanity.com/paper/2109.07075/) (2021)
- [Multiplayer Reach-Avoid Games via Pairwise Outcomes](https://mlsanity.com/paper/1611.00450/) (2016)
- [Distributional Reinforcement Learning based Integrated Decision Making and Control for Autonomous Surface Vehicles](https://mlsanity.com/paper/2412.09466/) (2024)
- [Multiplayer Homicidal Chauffeur Reach-Avoid Games: A Pursuit Enclosure Function Approach](https://mlsanity.com/paper/2311.02389/) (2023)

## Abstract (arXiv)

The target defense problem (TDP) for unmanned surface vehicles (USVs) concerns intercepting an adversarial USV before it breaches a designated target region, using one or more defending USVs. A particularly challenging scenario arises when the attacker exhibits superior maneuverability compared to the defenders, significantly complicating effective interception. To tackle this challenge, this letter introduces ARBoids, a novel adaptive residual reinforcement learning framework that integrates deep reinforcement learning (DRL) with the biologically inspired, force-based Boids model. Within this framework, the Boids model serves as a computationally efficient baseline policy for multi-agent coordination, while DRL learns a residual policy to adaptively refine and optimize the defenders' actions. The proposed approach is validated in a high-fidelity Gazebo simulation environment, demonstrating superior performance over traditional interception strategies, including pure force-based approaches and vanilla DRL policies. Furthermore, the learned policy exhibits strong adaptability to attackers with diverse maneuverability profiles, highlighting its robustness and generalization capability. The code of ARBoids will be released upon acceptance of this letter.