# Sim-to-Real RL for ASVs using SysID

arXiv: https://arxiv.org/abs/2610.12202 · HTML: https://mlsanity.com/paper/2610.12202/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in RL for robot navigation and UAV control, one of 15 directions in Deep reinforcement learning methods.
- Builds on 15 of its 29 references that are in the map.
- First posted on arXiv 8 Oct 2026.

## Placed in

- Deep reinforcement learning methods / [RL for robot navigation and UAV control](https://mlsanity.com/deeprl/rl-for-robot-navigation-and-uav-control/)

## Builds on (most cited first, 15 of 15)

- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [Domain Randomization for Transferring Deep Neural Networks from Simulation to the Real World](https://mlsanity.com/paper/1703.06907/) (2017)
- [Learning agile and dynamic motor skills for legged robots](https://mlsanity.com/paper/1901.08652/) (2019)
- [Closing the Sim-to-Real Loop: Adapting Simulation Randomization with Real World Experience](https://mlsanity.com/paper/1810.05687/) (2018)
- [BeyondMimic: From Motion Tracking to Versatile Humanoid Control via Guided Diffusion](https://mlsanity.com/paper/2508.08241/) (2025)
- [Deep Drone Racing: From Simulation to Reality with Domain Randomization](https://mlsanity.com/paper/1905.09727/) (2019)
- [skrl: Modular and Flexible Library for Reinforcement Learning](https://mlsanity.com/paper/2202.03825/) (2022)
- [Multi-Agent Manipulation via Locomotion using Hierarchical Sim2Real](https://mlsanity.com/paper/1908.05224/) (2019)
- [MarineGym: A High-Performance Reinforcement Learning Platform for Underwater Robotics](https://mlsanity.com/paper/2503.09203/) (2025)
- [Group Distributionally Robust Reinforcement Learning with Hierarchical Latent Variables](https://mlsanity.com/paper/2210.12262/) (2022)
- [Discrete-Time Hybrid Automata Learning: Legged Locomotion Meets Skateboarding](https://mlsanity.com/paper/2503.01842/) (2025)
- [Deep Reinforcement Learning Based Tracking Control of an Autonomous Surface Vessel in Natural Waters](https://mlsanity.com/paper/2302.08100/) (2023)
- [A Deep Reinforcement Learning Framework and Methodology for Reducing the Sim-to-Real Gap in ASV Navigation](https://mlsanity.com/paper/2407.08263/) (2024)
- [How To Train Your HERON](https://mlsanity.com/paper/2102.10357/) (2021)
- [Evaluating Robustness of Deep Reinforcement Learning for Autonomous Surface Vehicle Control in Field Tests](https://mlsanity.com/paper/2505.10033/) (2025)

## Abstract (arXiv)

Autonomous Surface Vehicles (ASVs) operating in dynamic marine environments require robust control policies for tasks such as path following and station keeping, making reinforcement learning (RL) a promising alternative to classical controllers. However, existing ASV simulators rarely support parallel environments for RL training. Such existing simulators require accurate hydrodynamic modeling from computational fluid dynamics solvers or towing tank tests for setting hydrodynamic parameters to address the sim-to-real gap. To address these challenges, we present an ASV simulator and accompanying pipeline that enables training policies starting from unknown vehicle dynamics. Our framework uses only a CAD model and brief set of open-water field trajectories for approximating and refining both hydrodynamic and thruster parameters. Real-world deployments on a BlueBoat ASV demonstrate successful zero-shot sim-to-real transfer in path following and station-keeping tasks without prior hydrodynamic and propeller information.