# Taming an autonomous surface vehicle for path following and collision avoidance using deep reinforcement learning

arXiv: https://arxiv.org/abs/1912.08578 · HTML: https://mlsanity.com/paper/1912.08578/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 18 Dec 2019.

## Built on by (most cited first, 9 of 9)

- [Physics guided neural networks for modelling of non-linear dynamics](https://mlsanity.com/paper/2205.06858/) (2022)
- [Hybrid analysis and modeling, eclecticism, and multifidelity computing toward digital twin revolution](https://mlsanity.com/paper/2103.14629/) (2021)
- [Survey of Deep Learning for Autonomous Surface Vehicles in the Marine Environment](https://mlsanity.com/paper/2210.08487/) (2022)
- [Risk-based implementation of COLREGs for autonomous surface vehicles using deep reinforcement learning](https://mlsanity.com/paper/2112.00115/) (2021)
- [Spatial-temporal recurrent reinforcement learning for autonomous ships](https://mlsanity.com/paper/2211.01004/) (2022)
- [Modular Control Architecture for Safe Marine Navigation: Reinforcement Learning and Predictive Safety Filters](https://mlsanity.com/paper/2312.01855/) (2023)
- [Depth-Constrained ASV Navigation with Deep RL and Limited Sensing](https://mlsanity.com/paper/2504.18253/) (2025)
- [Two-step dynamic obstacle avoidance](https://mlsanity.com/paper/2311.16841/) (2023)
- [Variational Autoencoders for exteroceptive perception in reinforcement learning-based collision avoidance](https://mlsanity.com/paper/2404.00623/) (2024)

## Abstract (arXiv)

In this article, we explore the feasibility of applying proximal policy optimization, a state-of-the-art deep reinforcement learning algorithm for continuous control tasks, on the dual-objective problem of controlling an underactuated autonomous surface vehicle to follow an a priori known path while avoiding collisions with non-moving obstacles along the way. The artificial intelligent agent, which is equipped with multiple rangefinder sensors for obstacle detection, is trained and evaluated in a challenging, stochastically generated simulation environment based on the OpenAI gym python toolkit. Notably, the agent is provided with real-time insight into its own reward function, allowing it to dynamically adapt its guidance strategy. Depending on its strategy, which ranges from radical path-adherence to radical obstacle avoidance, the trained agent achieves an episodic success rate between 84 and 100%.