# DivShift: Exploring Domain-Specific Distribution Shifts in Large-Scale, Volunteer-Collected Biodiversity Datasets

arXiv: https://arxiv.org/abs/2410.19816 · HTML: https://mlsanity.com/paper/2410.19816/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 16 of its 134 references that are in the map.
- First posted on arXiv 17 Oct 2024.

## Builds on (most cited first, 16 of 16)

- [LAION-5B: An open large-scale dataset for training next generation image-text models](https://mlsanity.com/paper/2210.08402/) (2022)
- [WILDS: A Benchmark of in-the-Wild Distribution Shifts](https://mlsanity.com/paper/2012.07421/) (2020)
- [Distributionally Robust Neural Networks for Group Shifts: On the Importance of Regularization for Worst-Case Generalization](https://mlsanity.com/paper/1911.08731/) (2019)
- [Recognition in Terra Incognita](https://mlsanity.com/paper/1807.04975/) (2018)
- [Benchmarking Representation Learning for Natural World Image Collections](https://mlsanity.com/paper/2103.16483/) (2021)
- [BioCLIP: A Vision Foundation Model for the Tree of Life](https://mlsanity.com/paper/2311.18803/) (2023)
- [The Majority Can Help The Minority: Context-rich Minority Oversampling for Long-tailed Classification](https://mlsanity.com/paper/2112.00412/) (2021)
- [Extending the WILDS Benchmark for Unsupervised Adaptation](https://mlsanity.com/paper/2112.05090/) (2021)
- [The iWildCam 2021 Competition Dataset](https://mlsanity.com/paper/2105.03494/) (2021)
- [Conditional Negative Sampling for Contrastive Learning of Visual Representations](https://mlsanity.com/paper/2010.02037/) (2020)
- [The Caltech Fish Counting Dataset: A Benchmark for Multiple-Object Tracking and Counting](https://mlsanity.com/paper/2207.09295/) (2022)
- [Tree-D Fusion: Simulation-Ready Tree Dataset from Single Images with Diffusion Priors](https://mlsanity.com/paper/2407.10330/) (2024)
- [BirdSAT: Cross-View Contrastive Masked Autoencoders for Bird Species Classification and Mapping](https://mlsanity.com/paper/2310.19168/) (2023)
- [Contrastive ground-level image and remote sensing pre-training improves representation learning for natural world imagery](https://mlsanity.com/paper/2409.19439/) (2024)
- [The GeoLifeCLEF 2020 Dataset](https://mlsanity.com/paper/2004.04192/) (2020)
- [Connect Later: Improving Fine-tuning for Robustness with Targeted Augmentations](https://mlsanity.com/paper/2402.03325/) (2024)

## Built on by (most cited first, 4 of 4)

- [Measuring Time-Series Dataset Similarity using Wasserstein Distance](https://mlsanity.com/paper/2507.22189/) (2025)
- [Unsupervised Urban Tree Biodiversity Mapping from Street-Level Imagery Using Spatially-Aware Visual Clustering](https://mlsanity.com/paper/2508.13814/) (2025)
- [BATIS: Bayesian Approaches for Targeted Improvement of Species Distribution Models](https://mlsanity.com/paper/2510.19749/) (2025)
- [SAGE: A sampling-aware global evaluation benchmark for species distribution modeling](https://mlsanity.com/paper/2609.31082/) (2026)

## Abstract (arXiv)

Large-scale, volunteer-collected datasets of community-identified natural world imagery like iNaturalist have enabled marked performance gains for fine-grained visual classification of species using machine learning methods. However, such data -- sometimes referred to as citizen science data -- are opportunistic and lack a structured sampling strategy. This volunteer-collected biodiversity data contains geographic, temporal, taxonomic, observers, and sociopolitical biases that can have significant effects on biodiversity model performance, but whose impacts are unclear for fine-grained species recognition performance. Here we introduce Diversity Shift (DivShift), a framework for quantifying the effects of domain-specific distribution shifts on machine learning model performance. To diagnose the performance effects of biases specific to volunteer-collected biodiversity data, we also introduce DivShift - North American West Coast (DivShift-NAWC), a curated dataset of almost 7.5 million iNaturalist images across the western coast of North America partitioned across five types of expert-verified bias. We compare species recognition performance across these bias partitions using a diverse variety of species- and ecosystem-focused accuracy metrics. We observe that these biases confound model performance less than expected from the underlying label distribution shift, and that more data leads to better model performance but the magnitude of these improvements are bias-specific. These findings imply that while the structure within natural world images provides generalization improvements for biodiversity monitoring tasks, the biases present in volunteer-collected biodiversity data can also affect model performance; thus these models should be used with caution in downstream biodiversity monitoring tasks.