mlsanityData to 14 Oct 2026

Learning to Swim: Reinforcement Learning for 6-DOF Control of Thruster-driven Autonomous Underwater Vehicles

arXiv 2410.00120 ↗First posted 30 Sep 2024cs.ROcited by 6

On the map: Deep reinforcement learning methods → RL for robot navigation and UAV control

Abstract (arXiv)

Controlling AUVs can be challenging because of the effect of complex non-linear hydrodynamic forces acting on the robot, which are significant in water and cannot be ignored. The problem is exacerbated for small AUVs for which the dynamics can change significantly with payload changes and deployments under different hydrodynamic conditions. The common approach to AUV control is a combination of passive stabilization with added buoyancy on top and weights on the bottom, and a PID controller tuned for simple and smooth motion primitives. However, the approach comes at the cost of sluggish controls and often the need to re-tune controllers with configuration changes. In this paper, we propose a fast (trainable in minutes), reinforcement learning-based approach for full 6 degree of freedom (DOF) control of a thruster-driven AUVs, taking 6-DOF command-conditioned inputs direct to thruster outputs. We present a new, highly parallelized simulator for underwater vehicle dynamics. We demonstrate this approach through zero-shot sim-to-real (with no tuning) transfer onto a real AUV that produces comparable results to hand-tuned PID controllers. Furthermore, we show that domain randomization on the simulator produces policies that are robust to small variations in vehicle's physical parameters.

Key facts. Cited by 6 papers in the map (data through 14 Oct 2026). Placed in RL for robot navigation and UAV control, one of 15 directions in Deep reinforcement learning methods. Builds on 21 of its 38 references that are in the map. First posted on arXiv 30 Sep 2024.

Builds on

21 of 38 references.

References · 21

  1. 2017Proximal Policy Optimization Algorithmscited by 15,215
  2. 2016OpenAI Gymcited by 1,736
  3. 2019Dream to Control: Learning Behaviors by Latent Imaginationcited by 1,653
  4. 2017Domain Randomization for Transferring Deep Neural Networks from Simulation to the Real Worldcited by 1,498
  5. 2021Isaac Gym: High Performance GPU-Based Physics Simulation For Robot Learningcited by 998
  6. 2020Learning Quadrupedal Locomotion over Challenging Terraincited by 604
  7. 2021Learning to Walk in Minutes Using Massively Parallel Deep Reinforcement Learningcited by 512
  8. 2019Learning agile and dynamic motor skills for legged robotscited by 504
  9. 2022DayDreamer: World Models for Physical Robot Learningcited by 400
  10. 2023Orbit: A Unified Simulation Framework for Interactive Robot Learning Environmentscited by 295
  11. 2020Learning Agile Robotic Locomotion Skills by Imitating Animalscited by 251
  12. 2022Rapid Locomotion via Reinforcement Learningcited by 168
9 more
  1. 2021Learning High-Speed Flight in the Wildcited by 128
  2. 2019Deep Drone Racing: From Simulation to Reality with Domain Randomizationcited by 76
  3. 2019Sim-to-(Multi)-Real: Transfer of Low-Level Robust Control Policies to Multiple Quadrotorscited by 45
  4. 2023Learning Vision-Based Bipedal Locomotion for Challenging Terraincited by 42
  5. 2023Learning to Fly in Secondscited by 31
  6. 2023UNav-Sim: A Visually Realistic Underwater Robotics Simulator and Synthetic Data-generation Frameworkcited by 17
  7. 2023CUREE: A Curious Underwater Robot for Ecosystem Explorationcited by 13
  8. 2022DAVE Aquatic Virtual Environment: Toward a General Underwater Robotics Simulatorcited by 11
  9. 2023From Concept to Field Tests: Accelerated Development of Multi-AUV Missions Using a High-Fidelity Faster-than-Real-Time Simulatorcited by 2

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 4; 2026: 2.

6 papers.

  1. 2025Never too Prim to Swim: An LLM-Enhanced RL-based Adaptive S-Surface Controller for AUVs under Extreme Sea Conditionscited by 4
  2. 2025Fast Policy Learning for 6-DOF Position Control of Underwater Vehiclescited by 3
  3. 2025Make Your AUV Adaptive: An Environment-Aware Reinforcement Learning Framework For Underwater Taskscited by 0
  4. 2025Learning to Dock: A Simulation-based Study on Closing the Sim2Real Gap in Autonomous Underwater Dockingcited by 0
  5. 2026Towards End to End Motion Planning and Execution for Autonomous Underwater Vehicles Using Reinforcement Learningcited by 0
  6. 2026AquaOrbit: Sim-to-Real Reinforcement Learning for Underwater Target Orbiting under Intermittent Visual Feedbackcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Toward 6-DOF Autonomous Underwater Vehicle Energy-Aware Position Control based on Deep Reinforcement Learning: Preliminary Results
  2. 2026CORL-AUV: CFD Oriented Reinforcement Learning for Autonomous Underwater Vehicles
  3. 2025Sim2Swim: Zero-Shot Velocity Control for Agile AUV Maneuvering in 3 Minutes
  4. 2025Ocean Diviner: A Diffusion-Augmented Reinforcement Learning Framework for AUV Robust Control in Underwater Tasks
  5. 2026A Unified Dynamics Framework for Reinforcement Learning and Classical Control of a Six-DOF Pipeline-Tracking ROV in NVIDIA Isaac Sim
  6. 2025When Motion Learns to Listen: Diffusion-Prior Lyapunov Actor-Critic Framework with LLM Guidance for Stable and Robust AUV Control in Underwater Tasks
  7. 2025EasyUUV: An LLM-Enhanced Universal and Lightweight Sim-to-Real Reinforcement Learning Framework for UUV Attitude Control ↗
  8. 2026Sim-to-Real RL for ASVs using SysID