# Video-based Hierarchical Species Classification for Longline Fishing Monitoring

arXiv: https://arxiv.org/abs/2102.03520 · HTML: https://mlsanity.com/paper/2102.03520/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Builds on 9 of its 22 references that are in the map.
- First posted on arXiv 6 Feb 2021.

## Builds on (most cited first, 9 of 9)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [Mask R-CNN](https://mlsanity.com/paper/1703.06870/) (2017)
- [Going Deeper with Convolutions](https://mlsanity.com/paper/1409.4842/) (2014)
- [HDLTex: Hierarchical Deep Learning for Text Classification](https://mlsanity.com/paper/1709.08267/) (2017)
- [B-CNN: Branch Convolutional Neural Network for Hierarchical Classification](https://mlsanity.com/paper/1709.09890/) (2017)
- [Solving Long-tailed Recognition with Deep Realistic Taxonomic Classifier](https://mlsanity.com/paper/2007.09898/) (2020)
- [HMIC: Hierarchical Medical Image Classification, A Deep Learning Approach](https://mlsanity.com/paper/2006.07187/) (2020)
- [Tracking Live Fish from Low-Contrast and Low-Frame-Rate Stereo Videos](https://mlsanity.com/paper/1504.03811/) (2015)

## Built on by (most cited first, 6 of 6)

- [The Fishnet Open Images Database: A Dataset for Fish Detection and Fine-Grained Categorization in Fisheries](https://mlsanity.com/paper/2106.09178/) (2021)
- [Absolute 3D Pose Estimation and Length Measurement of Severely Deformed Fish from Monocular Videos in Longline Fishing](https://mlsanity.com/paper/2102.04639/) (2021)
- [Multi-Object Tracking by Iteratively Associating Detections with Uniform Appearance for Trawl-Based Fishing Bycatch Monitoring](https://mlsanity.com/paper/2304.04816/) (2023)
- [Unsupervised Severely Deformed Mesh Reconstruction (DMR) from a Single-View Image](https://mlsanity.com/paper/2201.09373/) (2022)
- [HCIL: Hierarchical Class Incremental Learning for Longline Fishing Visual Monitoring](https://mlsanity.com/paper/2202.13018/) (2022)
- [ProTPS: Prototype-Guided Text Prompt Selection for Continual Learning](https://mlsanity.com/paper/2604.01116/) (2026)

## Abstract (arXiv)

The goal of electronic monitoring (EM) of longline fishing is to monitor the fish catching activities on fishing vessels, either for the regulatory compliance or catch counting. Hierarchical classification based on videos allows for inexpensive and efficient fish species identification of catches from longline fishing, where fishes are under severe deformation and self-occlusion during the catching process. More importantly, the flexibility of hierarchical classification mitigates the laborious efforts of human reviews by providing confidence scores in different hierarchical levels. Some related works either use cascaded models for hierarchical classification or make predictions per image or predict one overlapping hierarchical data structure of the dataset in advance. However, with a known non-overlapping hierarchical data structure provided by fisheries scientists, our method enforces the hierarchical data structure and introduces an efficient training and inference strategy for video-based fisheries data. Our experiments show that the proposed method outperforms the classic flat classification system significantly and our ablation study justifies our contributions in CNN model design, training strategy, and the video-based inference schemes for the hierarchical fish species classification task.