# Context-Driven Detection of Invertebrate Species in Deep-Sea Video

arXiv: https://arxiv.org/abs/2206.00718 · HTML: https://mlsanity.com/paper/2206.00718/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Builds on 13 of its 46 references that are in the map.
- First posted on arXiv 1 Jun 2022.

## Builds on (most cited first, 13 of 13)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks](https://mlsanity.com/paper/1506.01497/) (2015)
- [Fast R-CNN](https://mlsanity.com/paper/1504.08083/) (2015)
- [ByteTrack: Multi-Object Tracking by Associating Every Detection Box](https://mlsanity.com/paper/2110.06864/) (2021)
- [Object Detection with Deep Learning: A Review](https://mlsanity.com/paper/1807.05511/) (2018)
- [What's the Point: Semantic Segmentation with Point Supervision](https://mlsanity.com/paper/1506.02106/) (2015)
- [Weakly Supervised Learning of Instance Segmentation with Inter-pixel Relations](https://mlsanity.com/paper/1904.05044/) (2019)
- [Context R-CNN: Long Term Temporal Context for Per-Camera Object Detection](https://mlsanity.com/paper/1912.03538/) (2019)
- [FathomNet: A global image database for enabling artificial intelligence in the ocean](https://mlsanity.com/paper/2109.14646/) (2021)
- [PCAMs: Weakly Supervised Semantic Segmentation Using Point Supervision](https://mlsanity.com/paper/2007.05615/) (2020)
- [Underwater Fish Detection with Weak Multi-Domain Supervision](https://mlsanity.com/paper/1905.10708/) (2019)

## Built on by (most cited first, 6 of 6)

- [HUPE: Heuristic Underwater Perceptual Enhancement with Semantic Collaborative Learning](https://mlsanity.com/paper/2411.18296/) (2024)
- [GPT Self-Supervision for a Better Data Annotator](https://mlsanity.com/paper/2306.04349/) (2023)
- [LOCL: Learning Object-Attribute Composition using Localization](https://mlsanity.com/paper/2210.03780/) (2022)
- [RareSpot: Spotting Small and Rare Wildlife in Aerial Imagery with Multi-Scale Consistency and Context-Aware Augmentation](https://mlsanity.com/paper/2506.19087/) (2025)
- [Context-Matched Collage Generation for Underwater Invertebrate Detection](https://mlsanity.com/paper/2211.08479/) (2022)
- [RareSpot+: A Benchmark, Model, and Active Learning Framework for Small and Rare Wildlife in Aerial Imagery](https://mlsanity.com/paper/2604.20000/) (2026)

## Abstract (arXiv)

Each year, underwater remotely operated vehicles (ROVs) collect thousands of hours of video of unexplored ocean habitats revealing a plethora of information regarding biodiversity on Earth. However, fully utilizing this information remains a challenge as proper annotations and analysis require trained scientists time, which is both limited and costly. To this end, we present a Dataset for Underwater Substrate and Invertebrate Analysis (DUSIA), a benchmark suite and growing large-scale dataset to train, validate, and test methods for temporally localizing four underwater substrates as well as temporally and spatially localizing 59 underwater invertebrate species. DUSIA currently includes over ten hours of footage across 25 videos captured in 1080p at 30 fps by an ROV following pre planned transects across the ocean floor near the Channel Islands of California. Each video includes annotations indicating the start and end times of substrates across the video in addition to counts of species of interest. Some frames are annotated with precise bounding box locations for invertebrate species of interest, as seen in Figure 1. To our knowledge, DUSIA is the first dataset of its kind for deep sea exploration, with video from a moving camera, that includes substrate annotations and invertebrate species that are present at significant depths where sunlight does not penetrate. Additionally, we present the novel context-driven object detector (CDD) where we use explicit substrate classification to influence an object detection network to simultaneously predict a substrate and species class influenced by that substrate. We also present a method for improving training on partially annotated bounding box frames. Finally, we offer a baseline method for automating the counting of invertebrate species of interest.