# USIM and U0: A Vision-Language-Action Dataset and Model for General Underwater Robots

arXiv: https://arxiv.org/abs/2510.07869 · HTML: https://mlsanity.com/paper/2510.07869/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Placed in Generalist VLA models and benchmarks, one of 13 directions in Vision-language-action models.
- Builds on 21 of its 45 references that are in the map.
- First posted on arXiv 9 Oct 2025.

## Placed in

- Vision-language-action models / [Generalist VLA models and benchmarks](https://mlsanity.com/vla/generalist-vla-models-and-benchmarks/)

## Builds on (most cited first, 21 of 21)

- [RT-2: Vision-Language-Action Models Transfer Web Knowledge to Robotic Control](https://mlsanity.com/paper/2307.15818/) (2023)
- [OpenVLA: An Open-Source Vision-Language-Action Model](https://mlsanity.com/paper/2406.09246/) (2024)
- [π0: A Vision-Language-Action Flow Model for General Robot Control](https://mlsanity.com/paper/2410.24164/) (2024)
- [π0.5: a Vision-Language-Action Model with Open-World Generalization](https://mlsanity.com/paper/2504.16054/) (2025)
- [GR00T N1: An Open Foundation Model for Generalist Humanoid Robots](https://mlsanity.com/paper/2503.14734/) (2025)
- [Open X-Embodiment: Robotic Learning Datasets and RT-X Models](https://mlsanity.com/paper/2310.08864/) (2023)
- [DROID: A Large-Scale In-The-Wild Robot Manipulation Dataset](https://mlsanity.com/paper/2403.12945/) (2024)
- [On Evaluation of Embodied Navigation Agents](https://mlsanity.com/paper/1807.06757/) (2018)
- [AgiBot World Colosseo: A Large-scale Manipulation Platform for Scalable and Intelligent Embodied Systems](https://mlsanity.com/paper/2503.06669/) (2025)
- [An Underwater Image Enhancement Benchmark Dataset and Beyond](https://mlsanity.com/paper/1901.05495/) (2019)
- [HITTER: A HumanoId Table TEnnis Robot via Hierarchical Planning and Learning](https://mlsanity.com/paper/2508.21043/) (2025)
- [The Marine Debris Dataset for Forward-Looking Sonar Semantic Segmentation](https://mlsanity.com/paper/2108.06800/) (2021)
- [AQUALOC: An Underwater Dataset for Visual-Inertial-Pressure Localization](https://mlsanity.com/paper/1910.14532/) (2019)
- [Stonefish: Supporting Machine Learning Research in Marine Robotics](https://mlsanity.com/paper/2502.11887/) (2025)
- [SubPipe: A Submarine Pipeline Inspection Dataset for Segmentation and Visual-inertial Localization](https://mlsanity.com/paper/2401.17907/) (2024)
- [UIVNAV: Underwater Information-driven Vision-based Navigation via Imitation Learning](https://mlsanity.com/paper/2309.08806/) (2023)
- [AquaticVision: Benchmarking Visual SLAM in Underwater Environment with Events and Frames](https://mlsanity.com/paper/2505.03448/) (2025)
- [FishGym: A High-Performance Physics-based Simulation Framework for Underwater Robot Learning](https://mlsanity.com/paper/2206.01683/) (2022)
- [Distributed AI Agents for Cognitive Underwater Robot Autonomy](https://mlsanity.com/paper/2507.23735/) (2025)
- [Multi-Agent Generative Adversarial Interactive Self-Imitation Learning for AUV Formation Control and Obstacle Avoidance](https://mlsanity.com/paper/2401.11378/) (2024)
- [Self-Improving Autonomous Underwater Manipulation](https://mlsanity.com/paper/2410.18969/) (2024)

## Built on by (most cited first, 2 of 2)

- [Assessing Vision-Language Models for Perception in Autonomous Underwater Robotic Software](https://mlsanity.com/paper/2602.10655/) (2026)
- [AquaWAM: A Dynamics-aware World Action Model for Underwater Embodied Agents](https://mlsanity.com/paper/2609.33299/) (2026)

## Abstract (arXiv)

Underwater environments pose unique challenges for robotic navigation and manipulation. While existing research has primarily focused on task-specific methods, studies on general-purpose intelligence for multi-task execution remain scarce. To address this gap, we propose a unified framework for general-purpose underwater robots that integrates perception and action driven by language instructions. First, we develop a data synthesis pipeline to construct USIM, a simulation-based dataset which comprises over 905K frames from 2275 trajectories, totaling approximately 25 hours of BlueROV2 interactions. Furthermore, we propose U0, a vision-language-action (VLA) model capable of executing various tasks from obstacle-avoidance navigation to three-dimensional mobile manipulation. The model features a convolution-attention-based perception (CAP) module, which incorporates target pose estimation as an auxiliary task to explicitly bolster the model's spatial awareness. For evaluation, we establish a systematic assessment framework and an automated pipeline encompassing both offline metrics and online task execution. Experimental results demonstrate that the USIM dataset significantly empowers existing VLA models to adapt to underwater scenarios. Notably, our U0 model achieves state-of-the-art performance: it reduces the offline mean action prediction error to 0.0359 and achieves an overall online success rate of 43.1%, marking a 5.5% improvement over existing competitive baselines (below 37.6%), with navigation tasks reaching as high as 87.5%. These results validate the feasibility of general-purpose intelligence in underwater robotics, providing a foundation for scalable dataset synthesis and aquatic embodied agents.