# Risk-based implementation of COLREGs for autonomous surface vehicles using deep reinforcement learning

arXiv: https://arxiv.org/abs/2112.00115 · HTML: https://mlsanity.com/paper/2112.00115/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- Builds on 9 of its 70 references that are in the map.
- First posted on arXiv 30 Nov 2021.

## Builds on (most cited first, 9 of 9)

- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [Continuous control with deep reinforcement learning](https://mlsanity.com/paper/1509.02971/) (2015)
- [Asynchronous Methods for Deep Reinforcement Learning](https://mlsanity.com/paper/1602.01783/) (2016)
- [Exploring the Limitations of Behavior Cloning for Autonomous Driving](https://mlsanity.com/paper/1904.08980/) (2019)
- [COLREG-Compliant Collision Avoidance for Unmanned Surface Vehicle using Deep Reinforcement Learning](https://mlsanity.com/paper/2006.09540/) (2020)
- [Extended Object Tracking: Introduction, Overview and Applications](https://mlsanity.com/paper/1604.00970/) (2016)
- [Taming an autonomous surface vehicle for path following and collision avoidance using deep reinforcement learning](https://mlsanity.com/paper/1912.08578/) (2019)
- [A Survey of Deep Network Solutions for Learning Control in Robotics: From Reinforcement to Imitation](https://mlsanity.com/paper/1612.07139/) (2016)
- [Towards the Verification of Safety-critical Autonomous Systems in Dynamic Environments](https://mlsanity.com/paper/1612.04977/) (2016)

## Built on by (most cited first, 8 of 8)

- [Spatial-temporal recurrent reinforcement learning for autonomous ships](https://mlsanity.com/paper/2211.01004/) (2022)
- [2-Level Reinforcement Learning for Ships on Inland Waterways: Path Planning and Following](https://mlsanity.com/paper/2307.16769/) (2023)
- [Provable Traffic Rule Compliance in Safe Reinforcement Learning on the Open Sea](https://mlsanity.com/paper/2402.08502/) (2024)
- [Distributional Reinforcement Learning based Integrated Decision Making and Control for Autonomous Surface Vehicles](https://mlsanity.com/paper/2412.09466/) (2024)
- [Modular Control Architecture for Safe Marine Navigation: Reinforcement Learning and Predictive Safety Filters](https://mlsanity.com/paper/2312.01855/) (2023)
- [Aeolus Ocean -- A simulation environment for the autonomous COLREG-compliant navigation of Unmanned Surface Vehicles using Deep Reinforcement Learning and Maritime Object Detection](https://mlsanity.com/paper/2307.06688/) (2023)
- [Two-step dynamic obstacle avoidance](https://mlsanity.com/paper/2311.16841/) (2023)
- [A Large-Scale AIS Dataset from Finnish Water](https://mlsanity.com/paper/2609.12938/) (2026)

## Abstract (arXiv)

Autonomous systems are becoming ubiquitous and gaining momentum within the marine sector. Since the electrification of transport is happening simultaneously, autonomous marine vessels can reduce environmental impact, lower costs, and increase efficiency. Although close monitoring is still required to ensure safety, the ultimate goal is full autonomy. One major milestone is to develop a control system that is versatile enough to handle any weather and encounter that is also robust and reliable. Additionally, the control system must adhere to the International Regulations for Preventing Collisions at Sea (COLREGs) for successful interaction with human sailors. Since the COLREGs were written for the human mind to interpret, they are written in ambiguous prose and therefore not machine-readable or verifiable. Due to these challenges and the wide variety of situations to be tackled, classical model-based approaches prove complicated to implement and computationally heavy. Within machine learning (ML), deep reinforcement learning (DRL) has shown great potential for a wide range of applications. The model-free and self-learning properties of DRL make it a promising candidate for autonomous vessels. In this work, a subset of the COLREGs is incorporated into a DRL-based path following and obstacle avoidance system using collision risk theory. The resulting autonomous agent dynamically interpolates between path following and COLREG-compliant collision avoidance in the training scenario, isolated encounter situations, and AIS-based simulations of real-world scenarios.