# AquaOrbit: Sim-to-Real Reinforcement Learning for Underwater Target Orbiting under Intermittent Visual Feedback

arXiv: https://arxiv.org/abs/2609.24054 · HTML: https://mlsanity.com/paper/2609.24054/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in RL for robot navigation and UAV control, one of 15 directions in Deep reinforcement learning methods.
- Builds on 10 of its 24 references that are in the map.
- First posted on arXiv 21 Sep 2026.

## Placed in

- Deep reinforcement learning methods / [RL for robot navigation and UAV control](https://mlsanity.com/deeprl/rl-for-robot-navigation-and-uav-control/)

## Builds on (most cited first, 10 of 10)

- [Domain Randomization for Transferring Deep Neural Networks from Simulation to the Real World](https://mlsanity.com/paper/1703.06907/) (2017)
- [Sim-to-Real Transfer of Robotic Control with Dynamics Randomization](https://mlsanity.com/paper/1710.06537/) (2017)
- [AI Flow at the Network Edge](https://mlsanity.com/paper/2411.12469/) (2024)
- [MarineGym: A High-Performance Reinforcement Learning Platform for Underwater Robotics](https://mlsanity.com/paper/2503.09203/) (2025)
- [Learning to Swim: Reinforcement Learning for 6-DOF Control of Thruster-driven Autonomous Underwater Vehicles](https://mlsanity.com/paper/2410.00120/) (2024)
- [Enhancing Neural Video Compression of Static Scenes with Positive-Incentive Noise](https://mlsanity.com/paper/2603.06095/) (2026)
- [A reinforcement learning path planning approach for range-only underwater target localization with autonomous vehicles](https://mlsanity.com/paper/2301.06863/) (2023)
- [Sim-to-Real Transfer of Adaptive Control Parameters for AUV Stabilization under Current Disturbance](https://mlsanity.com/paper/2310.11075/) (2023)
- [Virtual Elastic Tether: a New Approach for Multi-agent Navigation in Confined Aquatic Environments](https://mlsanity.com/paper/2403.10629/) (2024)
- [Confined Space Underwater Positioning Using Collaborative Robots](https://mlsanity.com/paper/2510.27151/) (2025)

## Abstract (arXiv)

Intermittent visual loss disrupts target-relative feedback during underwater orbiting, making it difficult to maintain coordinated motion and reacquire a moving target. We present AquaOrbit, a reinforcement-learning controller with a recovery module for underwater target orbiting under interrupted visual feedback. During detection loss, the recovery module uses latched line-of-sight, roll, and depth references to support stabilization and target reacquisition. We train the controller in Isaac Sim with dynamics, observation, and vision-loss randomization. Evaluated without retraining in Gazebo/ROS2 under a different physics engine and perception perturbations, AquaOrbit completes 20/20 orbiting trials in each of the static- and moving-target conditions on an unseen variable-depth 3-D trajectory. In the moving-target condition, it reduces mean line-of-sight error by approximately 46% relative to a PID-based visual servoing controller with recovery while maintaining comparable path-tracking accuracy; removing the recovery module reduces completion to 9/20. Zero-shot physical deployment with fully onboard perception and control demonstrates elliptical, figure-eight, and variable-depth circular trajectories, including the latter two trajectory types absent from training. The robot maintains attitude stability during manual occlusions lasting up to 8s and reacquires the target within 2.5s in the reported attitude-induced field-of-view loss events.