mlsanityData to 14 Oct 2026

A Unified Dynamics Framework for Reinforcement Learning and Classical Control of a Six-DOF Pipeline-Tracking ROV in NVIDIA Isaac Sim

arXiv 2610.04949 ↗First posted 4 Oct 2026cs.RO cs.AI cs.SYcited by 0

On the map: Deep reinforcement learning methods → RL for robot navigation and UAV control

Abstract (arXiv)

Reinforcement learning controllers for underwater vehicles are usually trained against one physics representation and deployed against another, so reported performance does not always describe behavior outside training. This paper presents a pipeline-tracking architecture for a six-degree-of-freedom remotely operated vehicle (ROV) in which one Universal Scene Description (USD) scene supplies the real BlueROV2-Heavy mass, added-mass, damping, buoyancy, and thruster parameters to both halves of the system: a vectorized NumPy implementation of Fossen's marine-craft equations, and an interactive NVIDIA Isaac Sim deployment applying the identical equations as PhysX forces at every step. The Coriolis-centripetal term is the primary dynamics model in both branches; a controlled ablation on PPO and TRPO shows that including it does not destabilize either algorithm and modestly improves tracking, about 19 percent lower standoff RMS error for TRPO. Five reinforcement learning algorithms, PPO, soft actor-critic, TD3, DDPG, and TRPO, are trained against one environment, reward, and randomized evaluation harness through a checkpoint-compatibility layer scoring any policy with the same code. The pipeline is extended with six classical baselines, PID, sliding-mode, fuzzy logic, feedback linearization, model predictive control, and an adaptive neuro-fuzzy inference system, driven by the same guidance geometry and thruster allocation as the learned policies. Under Coriolis-enabled dynamics, PPO, TRPO, and feedback linearization reach the strongest combination of 100 percent success and competitive accuracy; PID, fuzzy control, and the neuro-fuzzy baseline also reach 100 percent success with looser tracking; DDPG and TD3 each show a specific, explainable failure mode rather than a general weakness of off-policy learning; and classical control remains a strong baseline against the best learned policies.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in RL for robot navigation and UAV control, one of 15 directions in Deep reinforcement learning methods. Builds on 11 of its 32 references that are in the map. First posted on arXiv 4 Oct 2026.

Builds on

11 of 32 references.

Closest ancestors · 1

Same direction, newest first

  1. 2025MarineGym: A High-Performance Reinforcement Learning Platform for Underwater Robotics12 Mar

Other references · 10

  1. 2017Proximal Policy Optimization Algorithmscited by 15,215
  2. 2015Continuous control with deep reinforcement learningcited by 3,407
  3. 2015Trust Region Policy Optimizationcited by 2,580
  4. 2018Addressing Function Approximation Error in Actor-Critic Methodscited by 2,417
  5. 2015High-Dimensional Continuous Control Using Generalized Advantage Estimationcited by 2,026
  6. 2017Domain Randomization for Transferring Deep Neural Networks from Simulation to the Real Worldcited by 1,498
  7. 2018Soft Actor-Critic Algorithms and Applicationscited by 1,240
  8. 2021Isaac Gym: High Performance GPU-Based Physics Simulation For Robot Learningcited by 998
  9. 2018GPU-Accelerated Robotic Simulation for Distributed Reinforcement Learningcited by 90
  10. 2023UNav-Sim: A Visually Realistic Underwater Robotics Simulator and Synthetic Data-generation Frameworkcited by 17

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025Fast Policy Learning for 6-DOF Position Control of Underwater Vehicles
  2. 2024Learning to Swim: Reinforcement Learning for 6-DOF Control of Thruster-driven Autonomous Underwater Vehicles
  3. 2026CORL-AUV: CFD Oriented Reinforcement Learning for Autonomous Underwater Vehicles
  4. 2025Sim2Swim: Zero-Shot Velocity Control for Agile AUV Maneuvering in 3 Minutes
  5. 2024MarineGym: Accelerated Training for Underwater Vehicles with High-Fidelity RL Simulation
  6. 2025Toward 6-DOF Autonomous Underwater Vehicle Energy-Aware Position Control based on Deep Reinforcement Learning: Preliminary Results
  7. 2026Towards End to End Motion Planning and Execution for Autonomous Underwater Vehicles Using Reinforcement Learning
  8. 2026Sim-to-Real RL for ASVs using SysID