# Ensembles of Vision Transformers as a New Paradigm for Automated Classification in Ecology

arXiv: https://arxiv.org/abs/2203.01726 · HTML: https://mlsanity.com/paper/2203.01726/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- Builds on 25 of its 82 references that are in the map.
- First posted on arXiv 3 Mar 2022.

## Builds on (most cited first, 25 of 25)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [Densely Connected Convolutional Networks](https://mlsanity.com/paper/1608.06993/) (2016)
- [EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks](https://mlsanity.com/paper/1905.11946/) (2019)
- [MobileNetV2: Inverted Residuals and Linear Bottlenecks](https://mlsanity.com/paper/1801.04381/) (2018)
- [Training data-efficient image transformers & distillation through attention](https://mlsanity.com/paper/2012.12877/) (2020)
- [Decoupled Weight Decay Regularization](https://mlsanity.com/paper/1711.05101/) (2017)
- [Do ImageNet Classifiers Generalize to ImageNet?](https://mlsanity.com/paper/1902.10811/) (2019)
- [Averaging Weights Leads to Wider Optima and Better Generalization](https://mlsanity.com/paper/1803.05407/) (2018)
- [TensorFlow: Large-Scale Machine Learning on Heterogeneous Distributed Systems](https://mlsanity.com/paper/1603.04467/) (2016)
- [Intriguing Properties of Vision Transformers](https://mlsanity.com/paper/2105.10497/) (2021)
- [A Survey on Deep Transfer Learning](https://mlsanity.com/paper/1808.01974/) (2018)
- [Revisiting the Calibration of Modern Neural Networks](https://mlsanity.com/paper/2106.07998/) (2021)
- [Towards Understanding Ensemble, Knowledge Distillation and Self-Distillation in Deep Learning](https://mlsanity.com/paper/2012.09816/) (2020)
- [ViTAE: Vision Transformer Advanced by Exploring Intrinsic Inductive Bias](https://mlsanity.com/paper/2106.03348/) (2021)
- [Vision Transformers are Robust Learners](https://mlsanity.com/paper/2105.07581/) (2021)
- [TransFG: A Transformer Architecture for Fine-grained Recognition](https://mlsanity.com/paper/2103.07976/) (2021)
- [Seeing biodiversity: perspectives in machine learning for wildlife conservation](https://mlsanity.com/paper/2110.12951/) (2021)
- [Optimal Regularization Can Mitigate Double Descent](https://mlsanity.com/paper/2003.01897/) (2020)
- [Double Trouble in Double Descent : Bias and Variance(s) in the Lazy Regime](https://mlsanity.com/paper/2003.01054/) (2020)
- [Towards Highly Accurate Coral Texture Images Classification Using Deep Convolutional Neural Networks and Data Augmentation](https://mlsanity.com/paper/1804.00516/) (2018)
- [Deep Learning Classification of Lake Zooplankton](https://mlsanity.com/paper/2108.05258/) (2021)
- [Florida Wildlife Camera Trap Dataset](https://mlsanity.com/paper/2106.12628/) (2021)
- [Deep learning for Plankton and Coral Classification](https://mlsanity.com/paper/1908.05489/) (2019)

## Built on by (most cited first, 10 of 10)

- [Survey of Automatic Plankton Image Recognition: Challenges, Existing Solutions and Future Perspectives](https://mlsanity.com/paper/2305.11739/) (2023)
- [Self-Supervised Pretraining for Fine-Grained Plankton Recognition](https://mlsanity.com/paper/2503.11341/) (2025)
- [Open-Set Plankton Recognition](https://mlsanity.com/paper/2503.11318/) (2025)
- [Deep learning for multi-label classification of coral conditions in the Indo-Pacific via underwater photogrammetry](https://mlsanity.com/paper/2403.05930/) (2024)
- [Hybrid quantum-classical convolutional neural network for phytoplankton classification](https://mlsanity.com/paper/2303.03707/) (2023)
- [Producing Plankton Classifiers that are Robust to Dataset Shift](https://mlsanity.com/paper/2401.14256/) (2024)
- [Remote Sensing Image Classification Using Deep Ensemble Learning](https://mlsanity.com/paper/2603.05844/) (2026)
- [Disentangled representations of microscopy images](https://mlsanity.com/paper/2506.20649/) (2025)
- [Cross-modal learning for plankton recognition](https://mlsanity.com/paper/2603.16427/) (2026)
- [WildFin: An In-the-Wild Dataset for Fish Behavioral Recognition](https://mlsanity.com/paper/2608.21281/) (2026)

## Abstract (arXiv)

Monitoring biodiversity is paramount to manage and protect natural resources. Collecting images of organisms over large temporal or spatial scales is a promising practice to monitor the biodiversity of natural ecosystems, providing large amounts of data with minimal interference with the environment. Deep learning models are currently used to automate classification of organisms into taxonomic units. However, imprecision in these classifiers introduces a measurement noise that is difficult to control and can significantly hinder the analysis and interpretation of data. {We overcome this limitation through ensembles of Data-efficient image Transformers (DeiTs), which not only are easy to train and implement, but also significantly outperform} the previous state of the art (SOTA). We validate our results on ten ecological imaging datasets of diverse origin, ranging from plankton to birds. On all the datasets, we achieve a new SOTA, with a reduction of the error with respect to the previous SOTA ranging from 29.35% to 100.00%, and often achieving performances very close to perfect classification. Ensembles of DeiTs perform better not because of superior single-model performances but rather due to smaller overlaps in the predictions by independent models and lower top-1 probabilities. This increases the benefit of ensembling, especially when using geometric averages to combine individual learners. While we only test our approach on biodiversity image datasets, our approach is generic and can be applied to any kind of images.