InfoTaxa: Information-Calibrated Label-Free Clustering for Fine-Grained Visual Taxonomy
Not on the map. Why?
Abstract (arXiv)
Label-free clustering of frozen pretrained visual embeddings offers a scalable route to biodiversity monitoring, but image-only fine-grained taxonomy exhibits a consistent coarse-to-fine failure mode: clusters recover broad taxonomic structure yet plateau at species level. We study this behaviour on BIOSCAN-5M through an information-calibrated clustering analysis. BioCLIP~2 features with UMAP and HDBSCAN reach 0.79 AMI at family and 0.67 at genus, substantially improving over the prior image baseline and remaining competitive with oracle-K, graph-based, and learned clustering heads on the same frozen features. To diagnose whether the remaining plateau is method-limited or information-limited, we introduce InfoTaxa, which combines clustering efficiency---the fraction of probe-estimated image information recovered by an unsupervised partition---with paired DNA as an audit signal only, not an inference input. The density pipeline recovers approximately 0.90 and 0.81 of the image-available information at order and family, respectively. Held-out late-fusion probes show that adding DNA to the image embedding reduces species-level prediction error by approximately two bits. Robustness analyses cover multiple image encoders, described-species and rare-class subsets, probe diagnostics, and held-out-species coarse-rank generalisation and same-species retrieval. Thus, in the tested setting, species-level label-free clustering is both clustering-limited and representation-limited: improved clustering may recover additional image-exposed structure, but cannot close the DNA-audited information gap alone.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 18 of its 32 references that are in the map. First posted on arXiv 15 Sep 2026.
Builds on
18 of 32 references.
References · 18
- 2023DINOv2: Learning Robust Visual Features without Supervisioncited by 7,974
- 2023Sigmoid Loss for Language Image Pre-Trainingcited by 3,109
- 2025DINOv3cited by 1,414
- 2018From Louvain to Leiden: guaranteeing well-connected communitiescited by 385
- 2023EVA-02: A Visual Representation for Neon Genesiscited by 381
- 2020SCAN: Learning to Classify Images without Labelscited by 352
- 2019On Mutual Information Maximization for Representation Learningcited by 316
- 2022Generalized Category Discoverycited by 211
- 2018Formal Limitations on the Measurement of Mutual Informationcited by 189
- 2023BioCLIP: A Vision Foundation Model for the Tree of Lifecited by 161
- 2022Parametric Classification for Generalized Category Discovery: A Baseline Studycited by 113
- 2019Efficient Parameter-free Clustering Using First Neighbor Relationscited by 97
6 more
- 2025BioCLIP 2: Emergent Properties from Scaling Hierarchical Contrastive Learningcited by 50
- 2023Exploring the Limits of Deep Image Clustering using Pretrained Modelscited by 33
- 2024BIOSCAN-5M: A Multimodal Dataset for Insect Biodiversitycited by 24
- 2024CLIBD: Bridging Vision and Genomics for Biodiversity Monitoring at Scalecited by 18
- 2024Let Go of Your Labels with Unsupervised Transfercited by 14
- 2024An Empirical Study into Clustering of Unseen Datasets with Self-Supervised Encoderscited by 5
Built on by
Not cited yet.