# Deep Reinforcement Learning Based Tracking Control of an Autonomous Surface Vessel in Natural Waters

arXiv: https://arxiv.org/abs/2302.08100 · HTML: https://mlsanity.com/paper/2302.08100/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 16 Feb 2023.

## Built on by (most cited first, 6 of 6)

- [A Deep Reinforcement Learning Framework and Methodology for Reducing the Sim-to-Real Gap in ASV Navigation](https://mlsanity.com/paper/2407.08263/) (2024)
- [Evaluating Robustness of Deep Reinforcement Learning for Autonomous Surface Vehicle Control in Field Tests](https://mlsanity.com/paper/2505.10033/) (2025)
- [Sim-to-Real Transfer and Robustness Evaluation of Reinforcement Learning Control with Integrated Perception on an ASV for Floating Waste Capture](https://mlsanity.com/paper/2605.02529/) (2026)
- [Cross-Platform Control for Autonomous Surface Vehicles via Adaptive Reinforcement Learning](https://mlsanity.com/paper/2607.02037/) (2026)
- [RL-Ballast: Ship Ballast Water Path Planning and Clog Prediction via Reinforcement Learning](https://mlsanity.com/paper/2607.04906/) (2026)
- [Sim-to-Real RL for ASVs using SysID](https://mlsanity.com/paper/2610.12202/) (2026)

## Abstract (arXiv)

Accurate control of autonomous marine robots still poses challenges due to the complex dynamics of the environment. In this paper, we propose a Deep Reinforcement Learning (DRL) approach to train a controller for autonomous surface vessel (ASV) trajectory tracking and compare its performance with an advanced nonlinear model predictive controller (NMPC) in real environments. Taking into account environmental disturbances (e.g., wind, waves, and currents), noisy measurements, and non-ideal actuators presented in the physical ASV, several effective reward functions for DRL tracking control policies are carefully designed. The control policies were trained in a simulation environment with diverse tracking trajectories and disturbances. The performance of the DRL controller has been verified and compared with the NMPC in both simulations with model-based environmental disturbances and in natural waters. Simulations show that the DRL controller has 53.33% lower tracking error than that of NMPC. Experimental results further show that, compared to NMPC, the DRL controller has 35.51% lower tracking error, indicating that DRL controllers offer better disturbance rejection in river environments than NMPC.