# Deep learning with self-supervision and uncertainty regularization to count fish in underwater images

arXiv: https://arxiv.org/abs/2104.14964 · HTML: https://mlsanity.com/paper/2104.14964/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- Builds on 21 of its 74 references that are in the map.
- First posted on arXiv 30 Apr 2021.

## Builds on (most cited first, 21 of 21)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks](https://mlsanity.com/paper/1506.01497/) (2015)
- [What Uncertainties Do We Need in Bayesian Deep Learning for Computer Vision?](https://mlsanity.com/paper/1703.04977/) (2017)
- [Meta Pseudo Labels](https://mlsanity.com/paper/2003.10580/) (2020)
- [The VIA Annotation Software for Images, Audio and Video](https://mlsanity.com/paper/1904.10699/) (2019)
- [Leveraging Unlabeled Data for Crowd Counting by Learning to Rank](https://mlsanity.com/paper/1803.03095/) (2018)
- [A Realistic Fish-Habitat Dataset to Evaluate Algorithms for Underwater Visual Analysis](https://mlsanity.com/paper/2008.12603/) (2020)
- [Perspective-Guided Convolution Networks for Crowd Counting](https://mlsanity.com/paper/1909.06966/) (2019)
- [CNN-based Density Estimation and Crowd Counting: A Survey](https://mlsanity.com/paper/2003.12783/) (2020)
- [Learning Spatial Awareness to Improve Crowd Counting](https://mlsanity.com/paper/1909.07057/) (2019)
- [VarGFaceNet: An Efficient Variable Group Convolutional Neural Network for Lightweight Face Recognition](https://mlsanity.com/paper/1910.04985/) (2019)
- [Crowd Counting with Decomposed Uncertainty](https://mlsanity.com/paper/1903.07427/) (2019)
- [Deep Learning Object Detection Methods for Ecological Camera Trap Data](https://mlsanity.com/paper/1803.10842/) (2018)
- [Dense Scale Network for Crowd Counting](https://mlsanity.com/paper/1906.09707/) (2019)
- [Active Multi-Label Crowd Consensus](https://mlsanity.com/paper/1911.02789/) (2019)
- [Underwater Fish Species Classification using Convolutional Neural Network and Deep Learning](https://mlsanity.com/paper/1805.10106/) (2018)
- [Underwater Fish Detection using Deep Learning for Water Power Applications](https://mlsanity.com/paper/1811.01494/) (2018)
- [Counting Fish and Dolphins in Sonar Images Using Deep Learning](https://mlsanity.com/paper/2007.12808/) (2020)
- [Deep-Learning-based Automated Palm Tree Counting and Geolocation in Large Farms from Aerial Geotagged Images](https://mlsanity.com/paper/2005.05269/) (2020)

## Built on by (most cited first, 9 of 9)

- [Computer Vision and Deep Learning for Fish Classification in Underwater Habitats: A Survey](https://mlsanity.com/paper/2203.06951/) (2022)
- [A data-centric approach for improving ambiguous labels with combined semi-supervised classification and clustering](https://mlsanity.com/paper/2106.16209/) (2021)
- [Sonar-based Deep Learning in Underwater Robotics: Overview, Robustness and Challenges](https://mlsanity.com/paper/2412.11840/) (2024)
- [Applications of Deep Learning in Fish Habitat Monitoring: A Tutorial and Survey](https://mlsanity.com/paper/2206.05394/) (2022)
- [Label Smarter, Not Harder: CleverLabel for Faster Annotation of Ambiguous Image Classification with Higher Quality](https://mlsanity.com/paper/2305.12811/) (2023)
- [Domain Adaptable Self-supervised Representation Learning on Remote Sensing Satellite Imagery](https://mlsanity.com/paper/2304.09874/) (2023)
- [Annotating Ambiguous Images: General Annotation Strategy for High-Quality Data with Real-World Biomedical Validation](https://mlsanity.com/paper/2306.12189/) (2023)
- [A deep neural network for multi-species fish detection using multiple acoustic cameras](https://mlsanity.com/paper/2109.10664/) (2021)
- [Fuzzy Overclustering: Semi-Supervised Classification of Fuzzy Labels with Overclustering and Inverse Cross-Entropy](https://mlsanity.com/paper/2110.06630/) (2021)

## Abstract (arXiv)

Effective conservation actions require effective population monitoring. However, accurately counting animals in the wild to inform conservation decision-making is difficult. Monitoring populations through image sampling has made data collection cheaper, wide-reaching and less intrusive but created a need to process and analyse this data efficiently. Counting animals from such data is challenging, particularly when densely packed in noisy images. Attempting this manually is slow and expensive, while traditional computer vision methods are limited in their generalisability. Deep learning is the state-of-the-art method for many computer vision tasks, but it has yet to be properly explored to count animals. To this end, we employ deep learning, with a density-based regression approach, to count fish in low-resolution sonar images. We introduce a large dataset of sonar videos, deployed to record wild mullet schools (Mugil liza), with a subset of 500 labelled images. We utilise abundant unlabelled data in a self-supervised task to improve the supervised counting task. For the first time in this context, by introducing uncertainty quantification, we improve model training and provide an accompanying measure of prediction uncertainty for more informed biological decision-making. Finally, we demonstrate the generalisability of our proposed counting framework through testing it on a recent benchmark dataset of high-resolution annotated underwater images from varying habitats (DeepFish). From experiments on both contrasting datasets, we demonstrate our network outperforms the few other deep learning models implemented for solving this task. By providing an open-source framework along with training data, our study puts forth an efficient deep learning template for crowd counting aquatic animals thereby contributing effective methods to assess natural populations from the ever-increasing visual data.