# Semi-Supervised Visual Tracking of Marine Animals using Autonomous Underwater Vehicles

arXiv: https://arxiv.org/abs/2302.07344 · HTML: https://mlsanity.com/paper/2302.07344/

## Key facts

- Cited by 16 papers in the map (data through 14 Oct 2026).
- Builds on 25 of its 57 references that are in the map.
- First posted on arXiv 14 Feb 2023.

## Builds on (most cited first, 25 of 25)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Simple Online and Realtime Tracking with a Deep Association Metric](https://mlsanity.com/paper/1703.07402/) (2017)
- [GOT-10k: A Large High-Diversity Benchmark for Generic Object Tracking in the Wild](https://mlsanity.com/paper/1810.11981/) (2018)
- [SiamRPN++: Evolution of Siamese Visual Tracking with Very Deep Networks](https://mlsanity.com/paper/1812.11703/) (2018)
- [Transformer Tracking](https://mlsanity.com/paper/2103.15436/) (2021)
- [YouTube-VOS: A Large-Scale Video Object Segmentation Benchmark](https://mlsanity.com/paper/1809.03327/) (2018)
- [Learning Discriminative Model Prediction for Tracking](https://mlsanity.com/paper/1904.07220/) (2019)
- [TrackingNet: A Large-Scale Dataset and Benchmark for Object Tracking in the Wild](https://mlsanity.com/paper/1803.10794/) (2018)
- [ATOM: Accurate Tracking by Overlap Maximization](https://mlsanity.com/paper/1811.07628/) (2018)
- [Fast Online Object Tracking and Segmentation: A Unifying Approach](https://mlsanity.com/paper/1812.05050/) (2018)
- [ECO: Efficient Convolution Operators for Tracking](https://mlsanity.com/paper/1611.09224/) (2016)
- [Probabilistic Regression for Visual Tracking](https://mlsanity.com/paper/2003.12565/) (2020)
- [Transformer Meets Tracker: Exploiting Temporal Context for Robust Visual Tracking](https://mlsanity.com/paper/2103.11681/) (2021)
- [Learning Multi-Domain Convolutional Neural Networks for Visual Tracking](https://mlsanity.com/paper/1510.07945/) (2015)
- [Distractor-aware Siamese Networks for Visual Object Tracking](https://mlsanity.com/paper/1808.06048/) (2018)
- [Need for Speed: A Benchmark for Higher Frame Rate Object Tracking](https://mlsanity.com/paper/1703.05884/) (2017)
- [LaSOT: A High-quality Large-scale Single Object Tracking Benchmark](https://mlsanity.com/paper/2009.03465/) (2020)
- [Learning Target Candidate Association to Keep Track of What Not to Track](https://mlsanity.com/paper/2103.16556/) (2021)
- [Return of the Devil in the Details: Delving Deep into Convolutional Nets](https://mlsanity.com/paper/1405.3531/) (2014)
- [The 2019 DAVIS Challenge on VOS: Unsupervised Multi-Object Segmentation](https://mlsanity.com/paper/1905.00737/) (2019)
- [Siamese Instance Search for Tracking](https://mlsanity.com/paper/1605.05863/) (2016)
- [A Realistic Fish-Habitat Dataset to Evaluate Algorithms for Underwater Visual Analysis](https://mlsanity.com/paper/2008.12603/) (2020)
- [Long-term Tracking in the Wild: A Benchmark](https://mlsanity.com/paper/1803.09502/) (2018)
- [FathomNet: A global image database for enabling artificial intelligence in the ocean](https://mlsanity.com/paper/2109.14646/) (2021)

## Built on by (most cited first, 16 of 16)

- [SeaSplat: Representing Underwater Scenes with 3D Gaussian Splatting and a Physically Grounded Image Formation Model](https://mlsanity.com/paper/2409.17345/) (2024)
- [WebUOT-1M: Advancing Deep Underwater Object Tracking with A Million-Scale Benchmark](https://mlsanity.com/paper/2405.19818/) (2024)
- [CUREE: A Curious Underwater Robot for Ecosystem Exploration](https://mlsanity.com/paper/2303.03943/) (2023)
- [Improving Underwater Visual Tracking With a Large Scale Dataset and Image Enhancement](https://mlsanity.com/paper/2308.15816/) (2023)
- [DeepSeeColor: Realtime Adaptive Color Correction for Autonomous Underwater Vehicles via Deep Learning Methods](https://mlsanity.com/paper/2303.04025/) (2023)
- [HUPE: Heuristic Underwater Perceptual Enhancement with Semantic Collaborative Learning](https://mlsanity.com/paper/2411.18296/) (2024)
- [AquaticCLIP: A Vision-Language Foundation Model for Underwater Scene Analysis](https://mlsanity.com/paper/2502.01785/) (2025)
- [RGB-Sonar Tracking Benchmark and Spatial Cross-Attention Transformer Tracker](https://mlsanity.com/paper/2406.07189/) (2024)
- [GPT Self-Supervision for a Better Data Annotator](https://mlsanity.com/paper/2306.04349/) (2023)
- [UWBench: A Comprehensive Vision-Language Benchmark for Underwater Understanding](https://mlsanity.com/paper/2510.18262/) (2025)
- [Mamba Capsule Routing Towards Part-Whole Relational Camouflaged Object Detection](https://mlsanity.com/paper/2410.03987/) (2024)
- [UVLM: Benchmarking Video Language Model for Underwater World Understanding](https://mlsanity.com/paper/2507.02373/) (2025)
- [Camouflaged Object Tracking: A Benchmark](https://mlsanity.com/paper/2408.13877/) (2024)
- [WaterClear-GS: Optical-Aware Gaussian Splatting for Underwater Reconstruction and Restoration](https://mlsanity.com/paper/2601.19753/) (2026)
- [MUOT_3M: A 3 Million Frame Multimodal Underwater Benchmark and the MUTrack Tracking Method](https://mlsanity.com/paper/2602.18006/) (2026)
- [CORL-AUV: CFD Oriented Reinforcement Learning for Autonomous Underwater Vehicles](https://mlsanity.com/paper/2607.09557/) (2026)

## Abstract (arXiv)

In-situ visual observations of marine organisms is crucial to developing behavioural understandings and their relations to their surrounding ecosystem. Typically, these observations are collected via divers, tags, and remotely-operated or human-piloted vehicles. Recently, however, autonomous underwater vehicles equipped with cameras and embedded computers with GPU capabilities are being developed for a variety of applications, and in particular, can be used to supplement these existing data collection mechanisms where human operation or tags are more difficult. Existing approaches have focused on using fully-supervised tracking methods, but labelled data for many underwater species are severely lacking. Semi-supervised trackers may offer alternative tracking solutions because they require less data than fully-supervised counterparts. However, because there are not existing realistic underwater tracking datasets, the performance of semi-supervised tracking algorithms in the marine domain is not well understood. To better evaluate their performance and utility, in this paper we provide (1) a novel dataset specific to marine animals located at http://warp.whoi.edu/vmat/, (2) an evaluation of state-of-the-art semi-supervised algorithms in the context of underwater animal tracking, and (3) an evaluation of real-world performance through demonstrations using a semi-supervised algorithm on-board an autonomous underwater vehicle to track marine animals in the wild.