# MarineGym: Accelerated Training for Underwater Vehicles with High-Fidelity RL Simulation

arXiv: https://arxiv.org/abs/2410.14117 · HTML: https://mlsanity.com/paper/2410.14117/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Placed in RL for robot navigation and UAV control, one of 15 directions in Deep reinforcement learning methods.
- Builds on 3 of its 9 references that are in the map.
- First posted on arXiv 18 Oct 2024.

## Placed in

- Deep reinforcement learning methods / [RL for robot navigation and UAV control](https://mlsanity.com/deeprl/rl-for-robot-navigation-and-uav-control/)

## Builds on (most cited first, 3 of 3)

- [GPU-Accelerated Robotic Simulation for Distributed Reinforcement Learning](https://mlsanity.com/paper/1810.05762/) (2018)
- [UNav-Sim: A Visually Realistic Underwater Robotics Simulator and Synthetic Data-generation Framework](https://mlsanity.com/paper/2310.11927/) (2023)
- [Online Approximate Optimal Station Keeping of a Marine Craft in the Presence of a Current](https://mlsanity.com/paper/1710.10511/) (2017)

## Built on by (most cited first, 1 of 1)

- [OceanSim: A GPU-Accelerated Underwater Robot Perception Simulation Framework](https://mlsanity.com/paper/2503.01074/) (2025)

## Abstract (arXiv)

Reinforcement Learning (RL) is a promising solution, allowing Unmanned Underwater Vehicles (UUVs) to learn optimal behaviors through trial and error. However, existing simulators lack efficient integration with RL methods, limiting training scalability and performance. This paper introduces MarineGym, a novel simulation framework designed to enhance RL training efficiency for UUVs by utilizing GPU acceleration. MarineGym offers a 10,000-fold performance improvement over real-time simulation on a single GPU, enabling rapid training of RL algorithms across multiple underwater tasks. Key features include realistic dynamic modeling of UUVs, parallel environment execution, and compatibility with popular RL frameworks like PyTorch and TorchRL. The framework is validated through four distinct tasks: station-keeping, circle tracking, helical tracking, and lemniscate tracking. This framework sets the stage for advancing RL in underwater robotics and facilitating efficient training in complex, dynamic environments.