# SeafloorAI: A Large-scale Vision-Language Dataset for Seafloor Geological Survey

arXiv: https://arxiv.org/abs/2411.00172 · HTML: https://mlsanity.com/paper/2411.00172/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Builds on 32 of its 94 references that are in the map.
- First posted on arXiv 31 Oct 2024.

## Builds on (most cited first, 30 of 32)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [Visual Instruction Tuning](https://mlsanity.com/paper/2304.08485/) (2023)
- [Densely Connected Convolutional Networks](https://mlsanity.com/paper/1608.06993/) (2016)
- [Scaling Up Visual and Vision-Language Representation Learning With Noisy Text Supervision](https://mlsanity.com/paper/2102.05918/) (2021)
- [TransUNet: Transformers Make Strong Encoders for Medical Image Segmentation](https://mlsanity.com/paper/2102.04306/) (2021)
- [LAION-400M: Open Dataset of CLIP-Filtered 400 Million Image-Text Pairs](https://mlsanity.com/paper/2111.02114/) (2021)
- [LLaVA-Med: Training a Large Language-and-Vision Assistant for Biomedicine in One Day](https://mlsanity.com/paper/2306.00890/) (2023)
- [Conceptual 12M: Pushing Web-Scale Image-Text Pre-Training To Recognize Long-Tail Visual Concepts](https://mlsanity.com/paper/2102.08981/) (2021)
- [LISA: Reasoning Segmentation via Large Language Model](https://mlsanity.com/paper/2308.00692/) (2023)
- [PathVQA: 30000+ Questions for Medical Visual Question Answering](https://mlsanity.com/paper/2003.10286/) (2020)
- [PMC-VQA: Visual Instruction Tuning for Medical Visual Question Answering](https://mlsanity.com/paper/2305.10415/) (2023)
- [GeoChat: Grounded Large Vision-Language Model for Remote Sensing](https://mlsanity.com/paper/2311.15826/) (2023)
- [Learning to Learn Single Domain Generalization](https://mlsanity.com/paper/2003.13216/) (2020)
- [SMIL: Multimodal Learning with Severely Missing Modality](https://mlsanity.com/paper/2103.05677/) (2021)
- [RSGPT: A Remote Sensing Vision Language Model and Benchmark](https://mlsanity.com/paper/2307.15266/) (2023)
- [RSVQA: Visual Question Answering for Remote Sensing Data](https://mlsanity.com/paper/2003.07333/) (2020)
- [Are Multimodal Transformers Robust to Missing Modality?](https://mlsanity.com/paper/2204.05454/) (2022)
- [Semantic Segmentation of Underwater Imagery: Dataset and Benchmark](https://mlsanity.com/paper/2004.01241/) (2020)
- [LISA++: An Improved Baseline for Reasoning Segmentation with Large Language Model](https://mlsanity.com/paper/2312.17240/) (2023)
- [The Marine Debris Dataset for Forward-Looking Sonar Semantic Segmentation](https://mlsanity.com/paper/2108.06800/) (2021)
- [Out-of-Domain Generalization from a Single Source: An Uncertainty Quantification Approach](https://mlsanity.com/paper/2108.02888/) (2021)
- [DEAL: Disentangle and Localize Concept-level Explanations for VLMs](https://mlsanity.com/paper/2407.14412/) (2024)
- [Machine Learning for Shipwreck Segmentation from Side Scan Sonar Imagery: Dataset and Benchmark](https://mlsanity.com/paper/2401.14546/) (2024)
- [A Dataset with Multibeam Forward-Looking Sonar for Underwater Object Detection](https://mlsanity.com/paper/2212.00352/) (2022)
- [What's Mine is Yours: Pretrained CNNs for Limited Training Sonar ATR](https://mlsanity.com/paper/1706.09858/) (2017)
- [Topology-aware Robust Optimization for Out-of-distribution Generalization](https://mlsanity.com/paper/2307.13943/) (2023)

## Built on by (most cited first, 6 of 6)

- [EarthSE: A Benchmark for Evaluating Earth Scientific Exploration Capability of LLMs](https://mlsanity.com/paper/2505.17139/) (2025)
- [BenthiCat: An opti-acoustic dataset for advancing benthic classification and habitat mapping](https://mlsanity.com/paper/2510.04876/) (2025)
- [AI Benchmark Democratization and Carpentry](https://mlsanity.com/paper/2512.11588/) (2025)
- [DREAM: Domain-aware Reasoning for Efficient Autonomous Underwater Monitoring](https://mlsanity.com/paper/2509.13666/) (2025)
- [An MLCommons Scientific Benchmarks Ontology](https://mlsanity.com/paper/2511.05614/) (2025)
- [MarineEval: Assessing the Marine Intelligence of Vision-Language Models](https://mlsanity.com/paper/2512.21126/) (2025)

## Abstract (arXiv)

A major obstacle to the advancements of machine learning models in marine science, particularly in sonar imagery analysis, is the scarcity of AI-ready datasets. While there have been efforts to make AI-ready sonar image dataset publicly available, they suffer from limitations in terms of environment setting and scale. To bridge this gap, we introduce SeafloorAI, the first extensive AI-ready datasets for seafloor mapping across 5 geological layers that is curated in collaboration with marine scientists. We further extend the dataset to SeafloorGenAI by incorporating the language component in order to facilitate the development of both vision- and language-capable machine learning models for sonar imagery. The dataset consists of 62 geo-distributed data surveys spanning 17,300 square kilometers, with 696K sonar images, 827K annotated segmentation masks, 696K detailed language descriptions and approximately 7M question-answer pairs. By making our data processing source code publicly available, we aim to engage the marine science community to enrich the data pool and inspire the machine learning community to develop more robust models. This collaborative approach will enhance the capabilities and applications of our datasets within both fields.