# CORL-AUV: CFD Oriented Reinforcement Learning for Autonomous Underwater Vehicles

arXiv: https://arxiv.org/abs/2607.09557 · HTML: https://mlsanity.com/paper/2607.09557/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in RL for robot navigation and UAV control, one of 15 directions in Deep reinforcement learning methods.
- Builds on 12 of its 32 references that are in the map.
- First posted on arXiv 10 Jul 2026.

## Placed in

- Deep reinforcement learning methods / [RL for robot navigation and UAV control](https://mlsanity.com/deeprl/rl-for-robot-navigation-and-uav-control/)

## Builds on (most cited first, 12 of 12)

- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [Learning to Walk in Minutes Using Massively Parallel Deep Reinforcement Learning](https://mlsanity.com/paper/2109.11978/) (2021)
- [Sim-to-(Multi)-Real: Transfer of Low-Level Robust Control Policies to Multiple Quadrotors](https://mlsanity.com/paper/1903.04628/) (2019)
- [Semi-Supervised Visual Tracking of Marine Animals using Autonomous Underwater Vehicles](https://mlsanity.com/paper/2302.07344/) (2023)
- [CUREE: A Curious Underwater Robot for Ecosystem Exploration](https://mlsanity.com/paper/2303.03943/) (2023)
- [Navigation in the Presence of Obstacles for an Agile Autonomous Underwater Vehicle](https://mlsanity.com/paper/1903.11750/) (2019)
- [Vision-Based Goal-Conditioned Policies for Underwater Navigation in the Presence of Obstacles](https://mlsanity.com/paper/2006.16235/) (2020)
- [FishGym: A High-Performance Physics-based Simulation Framework for Underwater Robot Learning](https://mlsanity.com/paper/2206.01683/) (2022)
- [Enhancing Efficiency and Propulsion in Bio-mimetic Robotic Fish through End-to-End Deep Reinforcement Learning](https://mlsanity.com/paper/2506.04627/) (2025)
- [Fast Policy Learning for 6-DOF Position Control of Underwater Vehicles](https://mlsanity.com/paper/2512.13359/) (2025)
- [Learning Agile Swimming: An End-to-End Approach without CPGs](https://mlsanity.com/paper/2409.10019/) (2024)
- [Sim2Swim: Zero-Shot Velocity Control for Agile AUV Maneuvering in 3 Minutes](https://mlsanity.com/paper/2512.08656/) (2025)

## Abstract (arXiv)

Precise control and positioning of autonomous underwater vehicles (AUVs) is critical for sampling, maintenance, and survey applications. Reinforcement learning (RL) offers rapid controller development while handling a range of deployment parameters via domain randomization (DR). However, DR is limited by the underlying simulation's capacity to model real physics such as drag, which is a large contributor to sim-to-real gaps. Computational fluid dynamics (CFD) provides high-fidelity drag models but has significant computational cost. Thus, in this paper we train surrogate approximations of CFD data of a given vehicle, providing drag estimates 700,000x faster than solving for drag at each time step with CFD within the RL training pipeline. We deploy a Proximal Policy Optimization (PPO) policy zero-shot on a 6-DOF AUV in which policy training is performed on these surrogate drag models (SDMs), providing analysis on zero-shot transfer, reward shaping sensitivity, and physical parameter perturbations. On 15 identical segments in the ocean, we observe median gains of 19% in mean tracking error and 31% in control effort compared to the equivalent inertia box model. Our SDM based RL controller better predicts zero-shot transfer and reaches the most waypoints across all three reward shaping choices that we evaluate. Finally, we add 0.907 kg to the vehicle perturbing the mass, center-of-mass, and center-of-buoyancy. The policy reaches 100% of waypoints only when the SDM drag model is used; the policy fails to reach any of the 15 waypoints when the other drag models are used.