# BIOSCAN-5M: A Multimodal Dataset for Insect Biodiversity

arXiv: https://arxiv.org/abs/2406.12723 · HTML: https://mlsanity.com/paper/2406.12723/

## Key facts

- Cited by 24 papers in the map (data through 14 Oct 2026).
- Builds on 39 of its 94 references that are in the map.
- First posted on arXiv 18 Jun 2024.

## Builds on (most cited first, 30 of 39)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [ImageNet Large Scale Visual Recognition Challenge](https://mlsanity.com/paper/1409.0575/) (2014)
- [End-to-End Object Detection with Transformers](https://mlsanity.com/paper/2005.12872/) (2020)
- [Masked Autoencoders Are Scalable Vision Learners](https://mlsanity.com/paper/2111.06377/) (2021)
- [Emerging Properties in Self-Supervised Vision Transformers](https://mlsanity.com/paper/2104.14294/) (2021)
- [UMAP: Uniform Manifold Approximation and Projection for Dimension Reduction](https://mlsanity.com/paper/1802.03426/) (2018)
- [Billion-scale similarity search with GPUs](https://mlsanity.com/paper/1702.08734/) (2017)
- [An Empirical Study of Training Self-Supervised Vision Transformers](https://mlsanity.com/paper/2104.02057/) (2021)
- [VICReg: Variance-Invariance-Covariance Regularization for Self-Supervised Learning](https://mlsanity.com/paper/2105.04906/) (2021)
- [Domain-Specific Language Model Pretraining for Biomedical Natural Language Processing](https://mlsanity.com/paper/2007.15779/) (2020)
- [BiomedCLIP: a multimodal biomedical foundation model pretrained from fifteen million scientific image-text pairs](https://mlsanity.com/paper/2303.00915/) (2023)
- [Hyena Hierarchy: Towards Larger Convolutional Language Models](https://mlsanity.com/paper/2302.10866/) (2023)
- [A Cookbook of Self-Supervised Learning](https://mlsanity.com/paper/2304.12210/) (2023)
- [Generalized Category Discovery](https://mlsanity.com/paper/2201.02609/) (2022)
- [Super-Convergence: Very Fast Training of Neural Networks Using Large Learning Rates](https://mlsanity.com/paper/1708.07120/) (2017)
- [Self-supervised Pretraining of Visual Features in the Wild](https://mlsanity.com/paper/2103.01988/) (2021)
- [Quilt-1M: One Million Image-Text Pairs for Histopathology](https://mlsanity.com/paper/2306.11207/) (2023)
- [HyenaDNA: Long-Range Genomic Sequence Modeling at Single Nucleotide Resolution](https://mlsanity.com/paper/2306.15794/) (2023)
- [Benchmarking Representation Learning for Natural World Image Collections](https://mlsanity.com/paper/2103.16483/) (2021)
- [BioCLIP: A Vision Foundation Model for the Tree of Life](https://mlsanity.com/paper/2311.18803/) (2023)
- [DNABERT-2: Efficient Foundation Model and Benchmark For Multi-Species Genome](https://mlsanity.com/paper/2306.15006/) (2023)
- [DeepWeeds: A Multiclass Weed Species Image Dataset for Deep Learning](https://mlsanity.com/paper/1810.05726/) (2018)
- [Towards a Visual-Language Foundation Model for Computational Pathology](https://mlsanity.com/paper/2307.12914/) (2023)
- [Contrastive language and vision learning of general fashion concepts](https://mlsanity.com/paper/2204.03972/) (2022)
- [A Step Towards Worldwide Biodiversity Assessment: The BIOSCAN-1M Insect Dataset](https://mlsanity.com/paper/2307.10455/) (2023)
- [Insect-Foundation: A Foundation Model and Large-scale 1M Dataset for Visual Insect Understanding](https://mlsanity.com/paper/2311.15206/) (2023)

## Built on by (most cited first, 24 of 24)

- [BioCLIP 2: Emergent Properties from Scaling Hierarchical Contrastive Learning](https://mlsanity.com/paper/2505.23883/) (2025)
- [Fish-Vista: A Multi-Purpose Dataset for Understanding & Identification of Traits from Images](https://mlsanity.com/paper/2407.08027/) (2024)
- [Reducing Aleatoric and Epistemic Uncertainty through Multi-modal Data Acquisition](https://mlsanity.com/paper/2501.18268/) (2025)
- [Enhancing DNA Foundation Models to Address Masking Inefficiencies](https://mlsanity.com/paper/2502.18405/) (2025)
- [Open-Insect: Benchmarking Open-Set Recognition of Novel Species in Biodiversity Monitoring](https://mlsanity.com/paper/2503.01691/) (2025)
- [A multi-modal dataset for insect biodiversity with imagery and DNA at the trap and individual level](https://mlsanity.com/paper/2507.06972/) (2025)
- [BioCAP: Exploiting Synthetic Captions Beyond Labels in Biological Foundation Models](https://mlsanity.com/paper/2510.20095/) (2025)
- [G2PDiffusion: Cross-Species Genotype-to-Phenotype Prediction via Evolutionary Diffusion](https://mlsanity.com/paper/2502.04684/) (2025)
- [Taxonomic Reasoning for Rare Arthropods: Combining Dense Image Captioning and RAG for Interpretable Classification](https://mlsanity.com/paper/2503.10886/) (2025)
- [BioCube: A Multimodal Dataset for Biodiversity Research](https://mlsanity.com/paper/2505.11568/) (2025)
- [TerraIncognita: A Dynamic Benchmark for Species Discovery Using Frontier Models](https://mlsanity.com/paper/2506.03182/) (2025)
- [Bridged Clustering: Semi-Supervised Sparse Bridging](https://mlsanity.com/paper/2510.07182/) (2025)
- [CHIPS: Efficient CLIP Adaptation via Curvature-aware Hybrid Influence-based Data Selection](https://mlsanity.com/paper/2511.18519/) (2025)
- [BarcodeMamba+: Advancing State-Space Models for Fungal Biodiversity Research](https://mlsanity.com/paper/2512.15931/) (2025)
- [AquaMonitor: A multimodal multi-view image sequence dataset for real-life aquatic invertebrate biodiversity monitoring](https://mlsanity.com/paper/2505.22065/) (2025)
- [BioAutoML-NAS: An End-to-End AutoML Framework for Multimodal Insect Classification via Neural Architecture Search on Large-Scale Biodiversity Data](https://mlsanity.com/paper/2510.05888/) (2025)
- [Hard Samples, Bad Labels: Robust Loss Functions That Know When to Back Off](https://mlsanity.com/paper/2511.16512/) (2025)
- [A continental-scale dataset of ground beetles with high-resolution images and validated morphological trait measurements](https://mlsanity.com/paper/2601.10687/) (2026)
- [Automatic Image-Level Morphological Trait Annotation for Organismal Images](https://mlsanity.com/paper/2604.01619/) (2026)
- [PestVL-Net: Enabling Multimodal Pest Learning via Fine-grained Vision-Language Interaction](https://mlsanity.com/paper/2604.17278/) (2026)
- [InfoTaxa: Information-Calibrated Label-Free Clustering for Fine-Grained Visual Taxonomy](https://mlsanity.com/paper/2609.17218/) (2026)
- [BarcodeMAE+: Rethinking Masked Pretraining and Global Representations for DNA Barcode Foundation Models](https://mlsanity.com/paper/2609.35877/) (2026)
- [CUBE: Class-conditioned Unknown Boundary Evidence for Open-Set Insect Species Recognition](https://mlsanity.com/paper/2610.13610/) (2026)
- [VisualWorldBench: A Fine-Grained Multi-Task Benchmark for Evaluating Visual World Knowledge in MLLMs](https://mlsanity.com/paper/2610.20083/) (2026)

## Abstract (arXiv)

As part of an ongoing worldwide effort to comprehend and monitor insect biodiversity, this paper presents the BIOSCAN-5M Insect dataset to the machine learning community and establish several benchmark tasks. BIOSCAN-5M is a comprehensive dataset containing multi-modal information for over 5 million insect specimens, and it significantly expands existing image-based biological datasets by including taxonomic labels, raw nucleotide barcode sequences, assigned barcode index numbers, geographical, and size information. We propose three benchmark experiments to demonstrate the impact of the multi-modal data types on the classification and clustering accuracy. First, we pretrain a masked language model on the DNA barcode sequences of the BIOSCAN-5M dataset, and demonstrate the impact of using this large reference library on species- and genus-level classification performance. Second, we propose a zero-shot transfer learning task applied to images and DNA barcodes to cluster feature embeddings obtained from self-supervised learning, to investigate whether meaningful clusters can be derived from these representation embeddings. Third, we benchmark multi-modality by performing contrastive learning on DNA barcodes, image data, and taxonomic information. This yields a general shared embedding space enabling taxonomic classification using multiple types of information and modalities. The code repository of the BIOSCAN-5M Insect dataset is available at https://github.com/bioscan-ml/BIOSCAN-5M.