# CaveSeg: Deep Semantic Segmentation and Scene Parsing for Autonomous Underwater Cave Exploration

arXiv: https://arxiv.org/abs/2309.11038 · HTML: https://mlsanity.com/paper/2309.11038/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Builds on 26 of its 54 references that are in the map.
- First posted on arXiv 20 Sep 2023.

## Builds on (most cited first, 26 of 26)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Swin Transformer: Hierarchical Vision Transformer using Shifted Windows](https://mlsanity.com/paper/2103.14030/) (2021)
- [Feature Pyramid Networks for Object Detection](https://mlsanity.com/paper/1612.03144/) (2016)
- [Fully Convolutional Networks for Semantic Segmentation](https://mlsanity.com/paper/1411.4038/) (2014)
- [DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFs](https://mlsanity.com/paper/1606.00915/) (2016)
- [Encoder-Decoder with Atrous Separable Convolution for Semantic Image Segmentation](https://mlsanity.com/paper/1802.02611/) (2018)
- [SegFormer: Simple and Efficient Design for Semantic Segmentation with Transformers](https://mlsanity.com/paper/2105.15203/) (2021)
- [Pyramid Scene Parsing Network](https://mlsanity.com/paper/1612.01105/) (2016)
- [Rich feature hierarchies for accurate object detection and semantic segmentation](https://mlsanity.com/paper/1311.2524/) (2013)
- [Unified Perceptual Parsing for Scene Understanding](https://mlsanity.com/paper/1807.10221/) (2018)
- [Segmenter: Transformer for Semantic Segmentation](https://mlsanity.com/paper/2105.05633/) (2021)
- [Image Segmentation Using Deep Learning: A Survey](https://mlsanity.com/paper/2001.05566/) (2020)
- [Semantic Image Segmentation with Deep Convolutional Nets and Fully Connected CRFs](https://mlsanity.com/paper/1412.7062/) (2014)
- [Context Encoding for Semantic Segmentation](https://mlsanity.com/paper/1803.08904/) (2018)
- [Semantic Segmentation of Underwater Imagery: Dataset and Benchmark](https://mlsanity.com/paper/2004.01241/) (2020)
- [FastFCN: Rethinking Dilated Convolution in the Backbone for Semantic Segmentation](https://mlsanity.com/paper/1903.11816/) (2019)
- [Simultaneous Enhancement and Super-Resolution of Underwater Imagery for Improved Visual Perception](https://mlsanity.com/paper/2002.01155/) (2020)
- [SVAM: Saliency-guided Visual Attention Modeling by Autonomous Underwater Robots](https://mlsanity.com/paper/2011.06252/) (2020)
- [SVIn2: An Underwater SLAM System using Sonar, Visual, Inertial, and Depth Sensor](https://mlsanity.com/paper/1810.03200/) (2018)
- [Experimental Comparison of Open Source Visual-Inertial-Based State Estimation Algorithms in the Underwater Domain](https://mlsanity.com/paper/1904.02215/) (2019)
- [Real-Time Dense 3D Mapping of Underwater Environments](https://mlsanity.com/paper/2304.02704/) (2023)
- [Navigation in the Presence of Obstacles for an Agile Autonomous Underwater Vehicle](https://mlsanity.com/paper/1903.11750/) (2019)
- [High Definition, Inexpensive, Underwater Mapping](https://mlsanity.com/paper/2203.05640/) (2022)
- [AquaVis: A Perception-Aware Autonomous Navigation Framework for Underwater Vehicles](https://mlsanity.com/paper/2110.01646/) (2021)
- [Fast Direct Stereo Visual SLAM](https://mlsanity.com/paper/2112.01890/) (2021)
- [Weakly Supervised Caveline Detection For AUV Navigation Inside Underwater Caves](https://mlsanity.com/paper/2303.03670/) (2023)

## Built on by (most cited first, 7 of 7)

- [MARIS: Marine Open-Vocabulary Instance Segmentation with Geometric Enhancement and Semantic Alignment](https://mlsanity.com/paper/2510.15398/) (2025)
- [ShareCMP: Polarization-Aware RGB-P Semantic Segmentation](https://mlsanity.com/paper/2312.03430/) (2023)
- [ODYSSEE: Oyster Detection Yielded by Sensor Systems on Edge Electronics](https://mlsanity.com/paper/2409.07003/) (2024)
- [AquaFuse: Waterbody Fusion for Physics Guided View Synthesis of Underwater Scenes](https://mlsanity.com/paper/2411.01119/) (2024)
- [UStyle: Waterbody Style Transfer of Underwater Scenes by Depth-Guided Feature Synthesis](https://mlsanity.com/paper/2503.11893/) (2025)
- [AquaBEV: Monocular Underwater BEV Occupancy with 3D Sonar Supervision](https://mlsanity.com/paper/2609.04411/) (2026)
- [UW-VOS: A Large-Scale Dataset for Underwater Video Object Segmentation](https://mlsanity.com/paper/2603.24006/) (2026)

## Abstract (arXiv)

In this paper, we present CaveSeg - the first visual learning pipeline for semantic segmentation and scene parsing for AUV navigation inside underwater caves. We address the problem of scarce annotated training data by preparing a comprehensive dataset for semantic segmentation of underwater cave scenes. It contains pixel annotations for important navigation markers (e.g. caveline, arrows), obstacles (e.g. ground plane and overhead layers), scuba divers, and open areas for servoing. Through comprehensive benchmark analyses on cave systems in USA, Mexico, and Spain locations, we demonstrate that robust deep visual models can be developed based on CaveSeg for fast semantic scene parsing of underwater cave environments. In particular, we formulate a novel transformer-based model that is computationally light and offers near real-time execution in addition to achieving state-of-the-art performance. Finally, we explore the design choices and implications of semantic segmentation for visual servoing by AUVs inside underwater caves. The proposed model and benchmark dataset open up promising opportunities for future research in autonomous underwater cave exploration and mapping.