# Self-Supervised Pretraining for Fine-Grained Plankton Recognition

arXiv: https://arxiv.org/abs/2503.11341 · HTML: https://mlsanity.com/paper/2503.11341/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 31 of its 67 references that are in the map.
- First posted on arXiv 14 Mar 2025.

## Builds on (most cited first, 30 of 31)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [A Simple Framework for Contrastive Learning of Visual Representations](https://mlsanity.com/paper/2002.05709/) (2020)
- [DINOv2: Learning Robust Visual Features without Supervision](https://mlsanity.com/paper/2304.07193/) (2023)
- [Momentum Contrast for Unsupervised Visual Representation Learning](https://mlsanity.com/paper/1911.05722/) (2019)
- [Masked Autoencoders Are Scalable Vision Learners](https://mlsanity.com/paper/2111.06377/) (2021)
- [Emerging Properties in Self-Supervised Vision Transformers](https://mlsanity.com/paper/2104.14294/) (2021)
- [Rethinking the Inception Architecture for Computer Vision](https://mlsanity.com/paper/1512.00567/) (2015)
- [Bootstrap your own latent: A new approach to self-supervised Learning](https://mlsanity.com/paper/2006.07733/) (2020)
- [Training data-efficient image transformers & distillation through attention](https://mlsanity.com/paper/2012.12877/) (2020)
- [Gaussian Error Linear Units (GELUs)](https://mlsanity.com/paper/1606.08415/) (2016)
- [BEiT: BERT Pre-Training of Image Transformers](https://mlsanity.com/paper/2106.08254/) (2021)
- [Barlow Twins: Self-Supervised Learning via Redundancy Reduction](https://mlsanity.com/paper/2103.03230/) (2021)
- [Accurate, Large Minibatch SGD: Training ImageNet in 1 Hour](https://mlsanity.com/paper/1706.02677/) (2017)
- [Big Self-Supervised Models are Strong Semi-Supervised Learners](https://mlsanity.com/paper/2006.10029/) (2020)
- [SimMIM: A Simple Framework for Masked Image Modeling](https://mlsanity.com/paper/2111.09886/) (2021)
- [Deep Networks with Stochastic Depth](https://mlsanity.com/paper/1603.09382/) (2016)
- [Deep Learning in Neural Networks: An Overview](https://mlsanity.com/paper/1404.7828/) (2014)
- [A Survey on Contrastive Self-supervised Learning](https://mlsanity.com/paper/2011.00362/) (2020)
- [BEiT v2: Masked Image Modeling with Vector-Quantized Visual Tokenizers](https://mlsanity.com/paper/2208.06366/) (2022)
- [BioCLIP: A Vision Foundation Model for the Tree of Life](https://mlsanity.com/paper/2311.18803/) (2023)
- [The effectiveness of MAE pre-pretraining for billion-scale pretraining](https://mlsanity.com/paper/2303.13496/) (2023)
- [A Realistic Evaluation of Semi-Supervised Learning for Fine-Grained Classification](https://mlsanity.com/paper/2104.00679/) (2021)
- [WHOI-Plankton- A Large Scale Fine Grained Visual Recognition Benchmark Dataset for Plankton Classification](https://mlsanity.com/paper/1510.00745/) (2015)
- [Beyond Cats and Dogs: Semi-supervised Classification of fuzzy labels with overclustering](https://mlsanity.com/paper/2012.01768/) (2020)
- [Ensembles of Vision Transformers as a New Paradigm for Automated Classification in Ecology](https://mlsanity.com/paper/2203.01726/) (2022)
- [MorphoCluster: Efficient Annotation of Plankton images by Clustering](https://mlsanity.com/paper/2005.01595/) (2020)
- [Deep Learning Classification of Lake Zooplankton](https://mlsanity.com/paper/2108.05258/) (2021)
- [Roll With the Punches: Expansion and Shrinkage of Soft Label Selection for Semi-supervised Fine-Grained Learning](https://mlsanity.com/paper/2312.12237/) (2023)
- [Survey of Automatic Plankton Image Recognition: Challenges, Existing Solutions and Future Perspectives](https://mlsanity.com/paper/2305.11739/) (2023)

## Built on by (most cited first, 4 of 4)

- [Planktonzilla: Multimodal dataset and models for understanding plankton ecosystems](https://mlsanity.com/paper/2606.00080/) (2026)
- [Cross-modal learning for plankton recognition](https://mlsanity.com/paper/2603.16427/) (2026)
- [Self-Supervised Federated Learning under Data Heterogeneity for Label-Scarce Diatom Classification](https://mlsanity.com/paper/2603.29633/) (2026)
- [Multimodal Taxonomic Conditioning for Generative Plankton Imagery](https://mlsanity.com/paper/2609.11673/) (2026)

## Abstract (arXiv)

Plankton recognition is an important computer vision problem due to plankton's essential role in ocean food webs and carbon capture, highlighting the need for species-level monitoring. However, this task is challenging due to its fine-grained nature and dataset shifts caused by different imaging instruments and varying species distributions. As new plankton image datasets are collected at an increasing pace, there is a need for general plankton recognition models that require minimal expert effort for data labeling. In this work, we study large-scale self-supervised pretraining for fine-grained plankton recognition. We first employ masked autoencoding and a large volume of diverse plankton image data to pretrain a general-purpose plankton image encoder. Then we utilize fine-tuning to obtain accurate plankton recognition models for new datasets with a very limited number of labeled training images. Our experiments show that self-supervised pretraining with diverse plankton data clearly increases plankton recognition accuracy compared to standard ImageNet pretraining when the amount of training data is limited. Moreover, the accuracy can be further improved when unlabeled target data is available and utilized during the pretraining.