# A Deep Reinforcement Learning Framework and Methodology for Reducing the Sim-to-Real Gap in ASV Navigation

arXiv: https://arxiv.org/abs/2407.08263 · HTML: https://mlsanity.com/paper/2407.08263/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Placed in RL for robot navigation and UAV control, one of 15 directions in Deep reinforcement learning methods.
- Builds on 9 of its 25 references that are in the map.
- First posted on arXiv 11 Jul 2024.

## Placed in

- Deep reinforcement learning methods / [RL for robot navigation and UAV control](https://mlsanity.com/deeprl/rl-for-robot-navigation-and-uav-control/)

## Builds on (most cited first, 9 of 9)

- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [Isaac Gym: High Performance GPU-Based Physics Simulation For Robot Learning](https://mlsanity.com/paper/2108.10470/) (2021)
- [Model-Reference Reinforcement Learning for Collision-Free Tracking Control of Autonomous Surface Vehicles](https://mlsanity.com/paper/2008.07240/) (2020)
- [Survey of Deep Learning for Autonomous Surface Vehicles in the Marine Environment](https://mlsanity.com/paper/2210.08487/) (2022)
- [Robust Unmanned Surface Vehicle Navigation with Distributional Reinforcement Learning](https://mlsanity.com/paper/2307.16240/) (2023)
- [Deep Reinforcement Learning Based Tracking Control of an Autonomous Surface Vessel in Natural Waters](https://mlsanity.com/paper/2302.08100/) (2023)
- [RANS: Highly-Parallelised Simulator for Reinforcement Learning based Autonomous Navigating Spacecrafts](https://mlsanity.com/paper/2310.07393/) (2023)
- [Station-keeping control of an unmanned surface vehicle exposed to current and wind disturbances](https://mlsanity.com/paper/1702.04941/) (2017)
- [How To Train Your HERON](https://mlsanity.com/paper/2102.10357/) (2021)

## Built on by (most cited first, 6 of 6)

- [RoboRAN: A Unified Robotics Framework for Reinforcement Learning-Based Autonomous Navigation](https://mlsanity.com/paper/2505.14526/) (2025)
- [Evaluating Robustness of Deep Reinforcement Learning for Autonomous Surface Vehicle Control in Field Tests](https://mlsanity.com/paper/2505.10033/) (2025)
- [A Unified Experimental Architecture for Informative Path Planning: from Simulation to Deployment with GuadalPlanner](https://mlsanity.com/paper/2602.10702/) (2026)
- [Sim2Sea: Sim-to-Real Policy Transfer for Maritime Vessel Navigation in Congested Waters](https://mlsanity.com/paper/2603.04057/) (2026)
- [Sim-to-Real Transfer and Robustness Evaluation of Reinforcement Learning Control with Integrated Perception on an ASV for Floating Waste Capture](https://mlsanity.com/paper/2605.02529/) (2026)
- [Sim-to-Real RL for ASVs using SysID](https://mlsanity.com/paper/2610.12202/) (2026)

## Abstract (arXiv)

Despite the increasing adoption of Deep Reinforcement Learning (DRL) for Autonomous Surface Vehicles (ASVs), there still remain challenges limiting real-world deployment. In this paper, we first integrate buoyancy and hydrodynamics models into a modern Reinforcement Learning framework to reduce training time. Next, we show how system identification coupled with domain randomization improves the RL agent performance and narrows the sim-to-real gap. Real-world experiments for the task of capturing floating waste show that our approach lowers energy consumption by 13.1% while reducing task completion time by 7.4%. These findings, supported by sharing our open-source implementation, hold the potential to impact the efficiency and versatility of ASVs, contributing to environmental conservation efforts.