Vision Transformers versus convolutional neural networks for fine-grained orchid genus identification in a species-rich, data-poor flora: a controlled benchmark on the Orchidaceae of New Guinea
Not on the map. Why?
Abstract (arXiv)
New Guinea is the world's richest island flora (~2,856 orchid species), yet most species are represented by only a handful of photographs, far fewer than direct species-level classification requires. Methods for fine-grained identification in such species-rich, data-poor floras are needed, and it remains unclear which backbone architecture and pretraining strategy best support them. We built a two-stage system that first predicts the genus of a query photograph, then retrieves visually similar reference images of candidate species using FAISS. We compared four pretrained backbones -- two Vision Transformers (ViTs; DINOv2, BioCLIP 2) and two CNNs (ConvNeXt V2-L, EfficientNetV2-L) -- fine-tuned under an identical protocol on a fixed, species-stratified partition of 16,701 photographs spanning 120 genera and 1,350 species, assessing accuracy, calibration, error structure, species retrieval, and open-set detection of novel genera. DINOv2 attained the best genus performance (macro top-1 66.9%, 95% CI 63.7-70.6; global top-1 88.9%); both ViTs outranked both CNNs, and general-purpose self-supervised pretraining (DINOv2) outperformed domain-matched biological pretraining (BioCLIP 2) by 7.1 points of macro top-1. Errors concentrated on two abundant genera acting as error attractors. DINOv2 embeddings achieved species Recall@5 of 86.6% and genus Recall@5 of 98.7%; temperature scaling reduced every backbone's Expected Calibration Error to about 0.03; and a distance-based open-set gate flagged unseen genera (mean AUROC 0.958). A self-supervised Vision-Transformer backbone combined with embedding retrieval is an effective, deployable strategy for fine-grained identification in species-rich, data-poor floras. The system is released as an open web application (the New Guinea Orchid Identifier), offering a practical template for other hyperdiverse, under-documented taxa.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 24 of its 46 references that are in the map. First posted on arXiv 21 Sep 2026.
Builds on
24 of 46 references.
References · 24
- 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
- 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
- 2021Emerging Properties in Self-Supervised Vision Transformerscited by 6,968
- 2017On Calibration of Modern Neural Networkscited by 4,960
- 2017Billion-scale similarity search with GPUscited by 2,497
- 2016A Baseline for Detecting Misclassified and Out-of-Distribution Examples in Neural Networkscited by 2,385
- 2019Class-Balanced Loss Based on Effective Number of Samplescited by 1,245
- 2021EfficientNetV2: Smaller Models and Faster Trainingcited by 1,135
- 2023ConvNeXt V2: Co-designing and Scaling ConvNets with Masked Autoencoderscited by 712
- 2021Do Vision Transformers See Like Convolutional Neural Networks?cited by 675
- 2015An Introduction to Convolutional Neural Networkscited by 514
- 2021ConViT: Improving Vision Transformers with Soft Convolutional Inductive Biasescited by 484
12 more
- 2021Deep Long-Tailed Learning: A Surveycited by 417
- 2021Revisiting the Calibration of Modern Neural Networkscited by 390
- 2021Open-Set Recognition: a Good Closed-Set Classifier is All You Need?cited by 353
- 2020How Well Do Self-Supervised Models Transfer?cited by 213
- 2023BioCLIP: A Vision Foundation Model for the Tree of Lifecited by 161
- 2021TransFG: A Transformer Architecture for Fine-grained Recognitioncited by 132
- 2025BioCLIP 2: Emergent Properties from Scaling Hierarchical Contrastive Learningcited by 50
- 2022Constructing Balance from Imbalance for Long-tailed Image Recognitioncited by 25
- 2023Two Sides of Miscalibration: Identifying Over and Under-Confidence Prediction for Network Calibrationcited by 14
- 2020Two-View Fine-grained Classification of Plant Speciescited by 4
- 2024Cooperative learning of Pl@ntNet's Artificial Intelligence algorithm: how does it work and how can we improve it?cited by 4
- 2025Open-Insect: Benchmarking Open-Set Recognition of Novel Species in Biodiversity Monitoringcited by 3
Built on by
Not cited yet.