# Assessing Vision-Language Models for Perception in Autonomous Underwater Robotic Software

arXiv: https://arxiv.org/abs/2602.10655 · HTML: https://mlsanity.com/paper/2602.10655/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in Embodied and 3D vision-language models, one of 16 directions in Multimodal LLMs / vision-language models.
- Builds on 21 of its 42 references that are in the map.
- First posted on arXiv 11 Feb 2026.

## Placed in

- Multimodal LLMs / vision-language models / [Embodied and 3D vision-language models](https://mlsanity.com/mllm/embodied-and-3d-vision-language-models/)

## Builds on (most cited first, 21 of 21)

- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learning](https://mlsanity.com/paper/1506.02142/) (2015)
- [Improved Baselines with Visual Instruction Tuning](https://mlsanity.com/paper/2310.03744/) (2023)
- [BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and Generation](https://mlsanity.com/paper/2201.12086/) (2022)
- [Simple and Scalable Predictive Uncertainty Estimation using Deep Ensembles](https://mlsanity.com/paper/1612.01474/) (2016)
- [Qwen Technical Report](https://mlsanity.com/paper/2309.16609/) (2023)
- [GR00T N1: An Open Foundation Model for Generalist Humanoid Robots](https://mlsanity.com/paper/2503.14734/) (2025)
- [A Review of Uncertainty Quantification in Deep Learning: Techniques, Applications and Challenges](https://mlsanity.com/paper/2011.06225/) (2020)
- [A Survey of Uncertainty in Deep Neural Networks](https://mlsanity.com/paper/2107.03342/) (2021)
- [A Survey on Vision-Language-Action Models for Embodied AI](https://mlsanity.com/paper/2405.14093/) (2024)
- [F-VLM: Open-Vocabulary Object Detection upon Frozen Vision and Language Models](https://mlsanity.com/paper/2209.15639/) (2022)
- [A Survey on Uncertainty Quantification of Large Language Models: Taxonomy, Open Research Challenges, and Future Directions](https://mlsanity.com/paper/2412.05563/) (2024)
- [TrashCan: A Semantically-Segmented Dataset towards Visual Detection of Marine Debris](https://mlsanity.com/paper/2007.08097/) (2020)
- [Uncertainty-Aware Evaluation for Vision-Language Models](https://mlsanity.com/paper/2402.14418/) (2024)
- [MarineGPT: Unlocking Secrets of Ocean to the Public](https://mlsanity.com/paper/2310.13596/) (2023)
- [Evaluating Uncertainty and Quality of Vision-Language-Action-enabled Robots](https://mlsanity.com/paper/2507.17049/) (2025)
- [AquaticCLIP: A Vision-Language Foundation Model for Underwater Scene Analysis](https://mlsanity.com/paper/2502.01785/) (2025)
- [Deep Learning Innovations for Underwater Waste Detection: An In-Depth Analysis](https://mlsanity.com/paper/2405.18299/) (2024)
- [UnderwaterVLA: Dual-brain Vision-Language-Action architecture for Autonomous Underwater Navigation](https://mlsanity.com/paper/2509.22441/) (2025)
- [MARS: Multi-Scale Adaptive Robotics Vision for Underwater Object Detection and Domain Generalization](https://mlsanity.com/paper/2312.15275/) (2023)
- [USIM and U0: A Vision-Language-Action Dataset and Model for General Underwater Robots](https://mlsanity.com/paper/2510.07869/) (2025)

## Abstract (arXiv)

Autonomous Underwater Robots (AURs) operate in challenging underwater environments, including low visibility and harsh water conditions. Such conditions present challenges for software engineers developing perception modules for the AUR software. To successfully carry out these tasks, deep learning has been incorporated into the AUR software to support its operations. However, the unique challenges of underwater environments pose difficulties for deep learning models, which often rely on labeled data that is scarce and noisy. This may undermine the trustworthiness of AUR software that relies on perception modules. Vision-Language Models (VLMs) offer promising solutions for AUR software as they generalize to unseen objects and remain robust in noisy conditions by inferring information from contextual cues. Despite this potential, their performance and uncertainty in underwater environments remain understudied from a software engineering perspective. Motivated by the needs of an industrial partner in assurance and risk management for maritime systems to assess the potential use of VLMs in this context, we present an empirical evaluation of VLM-based perception modules within the AUR software. We assess their ability to detect underwater trash by computing performance, uncertainty, and their relationship, to enable software engineers to select appropriate VLMs for their AUR software.