# UMI-Underwater: Learning Underwater Manipulation without Underwater Teleoperation

arXiv: https://arxiv.org/abs/2603.27012 · HTML: https://mlsanity.com/paper/2603.27012/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 29 of its 43 references that are in the map.
- First posted on arXiv 27 Mar 2026.

## Builds on (most cited first, 29 of 29)

- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [SAM 2: Segment Anything in Images and Videos](https://mlsanity.com/paper/2408.00714/) (2024)
- [Diffusion Policy: Visuomotor Policy Learning via Action Diffusion](https://mlsanity.com/paper/2303.04137/) (2023)
- [Vision Transformers for Dense Prediction](https://mlsanity.com/paper/2103.13413/) (2021)
- [Towards Robust Monocular Depth Estimation: Mixing Datasets for Zero-shot Cross-dataset Transfer](https://mlsanity.com/paper/1907.01341/) (2019)
- [Depth Anything V2](https://mlsanity.com/paper/2406.09414/) (2024)
- [Universal Manipulation Interface: In-The-Wild Robot Teaching Without In-The-Wild Robots](https://mlsanity.com/paper/2402.10329/) (2024)
- [Learning Hand-Eye Coordination for Robotic Grasping with Deep Learning and Large-Scale Data Collection](https://mlsanity.com/paper/1603.02199/) (2016)
- [An Underwater Image Enhancement Benchmark Dataset and Beyond](https://mlsanity.com/paper/1901.05495/) (2019)
- [Supersizing Self-supervision: Learning to Grasp from 50K Tries and 700 Robot Hours](https://mlsanity.com/paper/1509.06825/) (2015)
- [TossingBot: Learning to Throw Arbitrary Objects with Residual Physics](https://mlsanity.com/paper/1903.11239/) (2019)
- [AffordanceNet: An End-to-End Deep Learning Approach for Object Affordance Detection](https://mlsanity.com/paper/1709.07326/) (2017)
- [Learning Synergies between Pushing and Grasping with Self-supervised Deep Reinforcement Learning](https://mlsanity.com/paper/1803.09956/) (2018)
- [CoTracker3: Simpler and Better Point Tracking by Pseudo-Labelling Real Videos](https://mlsanity.com/paper/2410.11831/) (2024)
- [RoboCat: A Self-Improving Generalist Agent for Robotic Manipulation](https://mlsanity.com/paper/2306.11706/) (2023)
- [UMI on Legs: Making Manipulation Policies Mobile with Manipulation-Centric Whole-body Controllers](https://mlsanity.com/paper/2407.10353/) (2024)
- [DexUMI: Using Human Hand as the Universal Manipulation Interface for Dexterous Manipulation](https://mlsanity.com/paper/2505.21864/) (2025)
- [FastUMI: A Scalable and Hardware-Independent Universal Manipulation Interface with Dataset](https://mlsanity.com/paper/2409.19499/) (2024)
- [Where2Explore: Few-shot Affordance Learning for Unseen Novel Categories of Articulated Objects](https://mlsanity.com/paper/2309.07473/) (2023)
- [Diving Deeper into Underwater Image Enhancement: A Survey](https://mlsanity.com/paper/1907.07863/) (2019)
- [So You Think You Can Scale Up Autonomous Robot Data Collection?](https://mlsanity.com/paper/2411.01813/) (2024)
- [In-the-Wild Compliant Manipulation with UMI-FT](https://mlsanity.com/paper/2601.09988/) (2026)
- [UMI-on-Air: Embodiment-Aware Guidance for Embodiment-Agnostic Visuomotor Policies](https://mlsanity.com/paper/2510.02614/) (2025)
- [FastUMI-100K: Advancing Data-driven Robotic Manipulation with a Large-scale UMI-style Dataset](https://mlsanity.com/paper/2510.08022/) (2025)
- [MILES: Making Imitation Learning Easy with Self-Supervision](https://mlsanity.com/paper/2410.19693/) (2024)
- [MV-UMI: A Scalable Multi-View Interface for Cross-Embodiment Learning](https://mlsanity.com/paper/2509.18757/) (2025)
- [TacUMI: A Multi-Modal Universal Manipulation Interface for Contact-Rich Tasks](https://mlsanity.com/paper/2601.14550/) (2026)
- [Self-Improving Autonomous Underwater Manipulation](https://mlsanity.com/paper/2410.18969/) (2024)
- [PUGS: Perceptual Uncertainty for Grasp Selection in Underwater Environments](https://mlsanity.com/paper/2502.09824/) (2025)

## Abstract (arXiv)

Underwater robotic grasping is difficult due to degraded, highly variable imagery and the expense of collecting diverse underwater demonstrations. We introduce a system that (i) autonomously collects successful underwater grasp demonstrations via a self-supervised data collection pipeline and (ii) transfers grasp knowledge from on-land human demonstrations through a depth-based affordance representation that bridges the on-land-to-underwater domain gap and is robust to lighting and color shift. An affordance model trained on on-land handheld demonstrations is deployed underwater zero-shot via geometric alignment, and an affordance-conditioned diffusion policy is then trained on underwater demonstrations to generate control actions. In pool experiments, our approach improves grasping performance and robustness to background shifts, and enables generalization to objects seen only in on-land data, outperforming RGB-only baselines. Code, videos, and additional results are available at https://umi-under-water.github.io.