# Decentralized Multi-Robot Navigation for Autonomous Surface Vehicles with Distributional Reinforcement Learning

arXiv: https://arxiv.org/abs/2402.11799 · HTML: https://mlsanity.com/paper/2402.11799/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Placed in RL for robot navigation and UAV control, one of 15 directions in Deep reinforcement learning methods.
- Builds on 15 of its 41 references that are in the map.
- First posted on arXiv 19 Feb 2024.

## Placed in

- Deep reinforcement learning methods / [RL for robot navigation and UAV control](https://mlsanity.com/deeprl/rl-for-robot-navigation-and-uav-control/)

## Builds on (most cited first, 15 of 15)

- [A Distributional Perspective on Reinforcement Learning](https://mlsanity.com/paper/1707.06887/) (2017)
- [Implicit Quantile Networks for Distributional Reinforcement Learning](https://mlsanity.com/paper/1806.06923/) (2018)
- [Crowd-Robot Interaction: Crowd-aware Robot Navigation with Attention-based Deep Reinforcement Learning](https://mlsanity.com/paper/1809.08835/) (2018)
- [Socially Aware Motion Planning with Deep Reinforcement Learning](https://mlsanity.com/paper/1703.08862/) (2017)
- [Decentralized Non-communicating Multiagent Collision Avoidance with Deep Reinforcement Learning](https://mlsanity.com/paper/1609.07845/) (2016)
- [Motion Planning Among Dynamic, Decision-Making Agents with Deep Reinforcement Learning](https://mlsanity.com/paper/1805.01956/) (2018)
- [Towards Optimally Decentralized Multi-Robot Collision Avoidance via Deep Reinforcement Learning](https://mlsanity.com/paper/1709.10082/) (2017)
- [Relational Graph Learning for Crowd Navigation](https://mlsanity.com/paper/1909.13165/) (2019)
- [Reinforcement Learned Distributed Multi-Robot Navigation with Reciprocal Velocity Obstacle Shaped Rewards](https://mlsanity.com/paper/2203.10229/) (2022)
- [Risk-Conditioned Distributional Soft Actor-Critic for Risk-Sensitive Navigation](https://mlsanity.com/paper/2104.03111/) (2021)
- [Minimizing Safety Interference for Safe and Comfortable Automated Driving with Distributional Reinforcement Learning](https://mlsanity.com/paper/2107.07316/) (2021)
- [Robust Unmanned Surface Vehicle Navigation with Distributional Reinforcement Learning](https://mlsanity.com/paper/2307.16240/) (2023)
- [Adaptive Risk-Tendency: Nano Drone Navigation in Cluttered Environments with Distributional Reinforcement Learning](https://mlsanity.com/paper/2203.14749/) (2022)
- [Multi-Robot Data Gathering Under Buffer Constraints and Intermittent Communication](https://mlsanity.com/paper/1706.02092/) (2017)
- [Robust Route Planning with Distributional Reinforcement Learning in a Stochastic Road Network Environment](https://mlsanity.com/paper/2304.09996/) (2023)

## Built on by (most cited first, 4 of 4)

- [Distributional Reinforcement Learning based Integrated Decision Making and Control for Autonomous Surface Vehicles](https://mlsanity.com/paper/2412.09466/) (2024)
- [AsyncShield: A Plug-and-Play Edge Adapter for Asynchronous Cloud-based VLA Navigation](https://mlsanity.com/paper/2604.24086/) (2026)
- [MarineFormer: A Spatio-Temporal Attention Model for USV Navigation in Dynamic Marine Environments](https://mlsanity.com/paper/2410.13973/) (2024)
- [TERL: Large-Scale Multi-Target Encirclement Using Transformer-Enhanced Reinforcement Learning](https://mlsanity.com/paper/2503.12395/) (2025)

## Abstract (arXiv)

Collision avoidance algorithms for Autonomous Surface Vehicles (ASV) that follow the Convention on the International Regulations for Preventing Collisions at Sea (COLREGs) have been proposed in recent years. However, it may be difficult and unsafe to follow COLREGs in congested waters, where multiple ASVs are navigating in the presence of static obstacles and strong currents, due to the complex interactions. To address this problem, we propose a decentralized multi-ASV collision avoidance policy based on Distributional Reinforcement Learning, which considers the interactions among ASVs as well as with static obstacles and current flows. We evaluate the performance of the proposed Distributional RL based policy against a traditional RL-based policy and two classical methods, Artificial Potential Fields (APF) and Reciprocal Velocity Obstacles (RVO), in simulation experiments, which show that the proposed policy achieves superior performance in navigation safety, while requiring minimal travel time and energy. A variant of our framework that automatically adapts its risk sensitivity is also demonstrated to improve ASV safety in highly congested environments.