# BirdSet: A Large-Scale Dataset for Audio Classification in Avian Bioacoustics

arXiv: https://arxiv.org/abs/2403.10380 · HTML: https://mlsanity.com/paper/2403.10380/

## Key facts

- Cited by 36 papers in the map (data through 14 Oct 2026).
- Builds on 34 of its 92 references that are in the map.
- First posted on arXiv 15 Mar 2024.

## Builds on (most cited first, 30 of 34)

- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks](https://mlsanity.com/paper/1905.11946/) (2019)
- [mixup: Beyond Empirical Risk Minimization](https://mlsanity.com/paper/1710.09412/) (2017)
- [GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understanding](https://mlsanity.com/paper/1804.07461/) (2018)
- [wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representations](https://mlsanity.com/paper/2006.11477/) (2020)
- [A ConvNet for the 2020s](https://mlsanity.com/paper/2201.03545/) (2022)
- [A Baseline for Detecting Misclassified and Out-of-Distribution Examples in Neural Networks](https://mlsanity.com/paper/1610.02136/) (2016)
- [Datasheets for Datasets](https://mlsanity.com/paper/1803.09010/) (2018)
- [AugMix: A Simple Data Processing Method to Improve Robustness and Uncertainty](https://mlsanity.com/paper/1912.02781/) (2019)
- [VGGSound: A Large-scale Audio-Visual Dataset](https://mlsanity.com/paper/2004.14368/) (2020)
- [AST: Audio Spectrogram Transformer](https://mlsanity.com/paper/2104.01778/) (2021)
- [Datasets: A Community Library for Natural Language Processing](https://mlsanity.com/paper/2109.02846/) (2021)
- [BEATs: Audio Pre-Training with Acoustic Tokenizers](https://mlsanity.com/paper/2212.09058/) (2022)
- [FSD50K: An Open Dataset of Human-Labeled Sound Events](https://mlsanity.com/paper/2010.00475/) (2020)
- [Masked Autoencoders that Listen](https://mlsanity.com/paper/2207.06405/) (2022)
- [Speaker Recognition from Raw Waveform with SincNet](https://mlsanity.com/paper/1808.00158/) (2018)
- [Plex: Towards Reliability using Pretrained Large Model Extensions](https://mlsanity.com/paper/2207.07411/) (2022)
- [Deep Learning for Audio Signal Processing](https://mlsanity.com/paper/1905.00078/) (2019)
- [Computational bioacoustics with deep learning: a review and roadmap](https://mlsanity.com/paper/2112.06725/) (2021)
- [EAT: Self-Supervised Pre-Training with Efficient Audio Transformer](https://mlsanity.com/paper/2401.03497/) (2024)
- [Epic-Sounds: A Large-scale Dataset of Actions That Sound](https://mlsanity.com/paper/2302.00646/) (2023)
- [Automatic acoustic detection of birds through deep learning: the first Bird Audio Detection challenge](https://mlsanity.com/paper/1807.05812/) (2018)
- [BEANS: The Benchmark of Animal Sounds](https://mlsanity.com/paper/2210.12300/) (2022)
- [Improving Bird Classification with Unsupervised Sound Separation](https://mlsanity.com/paper/2110.03209/) (2021)
- [MetaAudio: A Few-Shot Audio Classification Benchmark](https://mlsanity.com/paper/2204.02121/) (2022)
- [BIRB: A Generalization Benchmark for Information Retrieval in Bioacoustics](https://mlsanity.com/paper/2312.07439/) (2023)
- [NIPS4Bplus: a richly annotated birdsong audio dataset](https://mlsanity.com/paper/1811.02275/) (2018)
- [ActiveGLAE: A Benchmark for Deep Active Learning with Transformers](https://mlsanity.com/paper/2306.10087/) (2023)
- [SONYC-UST-V2: An Urban Sound Tagging Dataset with Spatiotemporal Context](https://mlsanity.com/paper/2009.05188/) (2020)
- [Who calls the shots? Rethinking Few-Shot Learning for Audio](https://mlsanity.com/paper/2110.09600/) (2021)

## Built on by (most cited first, 30 of 36)

- [Can Masked Autoencoders Also Listen to Birds?](https://mlsanity.com/paper/2504.12880/) (2025)
- [Perch 2.0: The Bittern Lesson for Bioacoustics](https://mlsanity.com/paper/2508.04665/) (2025)
- [animal2vec and MeerKAT: A self-supervised transformer for rare-event raw audio input and a large-scale reference dataset for bioacoustics](https://mlsanity.com/paper/2406.01253/) (2024)
- [Foundation Models for Bioacoustics -- a Comparative Review](https://mlsanity.com/paper/2508.01277/) (2025)
- [Towards Deep Active Learning in Avian Bioacoustics](https://mlsanity.com/paper/2406.18621/) (2024)
- [Massive Sound Embedding Benchmark (MSEB)](https://mlsanity.com/paper/2602.07143/) (2026)
- [Unmute the Patch Tokens: Rethinking Probing in Multi-Label Audio Classification](https://mlsanity.com/paper/2509.24901/) (2025)
- [AVEX: What Matters for Animal Vocalization Encoding](https://mlsanity.com/paper/2508.11845/) (2025)
- [Finding Needles in the Haystack: Transductive Active Labeling in Ecology](https://mlsanity.com/paper/2606.03821/) (2026)
- [WoW-Bench: Evaluating Fine-Grained Acoustic Perception in Audio-Language Models via Marine Mammal Vocalizations](https://mlsanity.com/paper/2508.20976/) (2025)
- [Linear probing enables Ship-Radiated Noise recognition with pretrained audio embeddings](https://mlsanity.com/paper/2601.08358/) (2026)
- [Determinantal point process sampling for bioacoustic active learning](https://mlsanity.com/paper/2607.06063/) (2026)
- [MetaPerch: Learning from metadata for bioacoustics foundation models](https://mlsanity.com/paper/2607.14072/) (2026)
- [Localized time-frequency representation learning for bioacoustic classification in complex soundscapes](https://mlsanity.com/paper/2502.13440/) (2025)
- [Uncertainty Calibration of Multi-Label Bird Sound Classifiers](https://mlsanity.com/paper/2511.08261/) (2025)
- [Data-Efficient Self-Supervised Algorithms for Fine-Grained Birdsong Analysis](https://mlsanity.com/paper/2511.12158/) (2025)
- [WhAM: Towards A Translative Model of Sperm Whale Vocalization](https://mlsanity.com/paper/2512.02206/) (2025)
- [Multi-layer attentive probing improves transfer of audio representations for bioacoustics](https://mlsanity.com/paper/2605.10494/) (2026)
- [Masked Autoencoders with Limited Data: Does It Work? A Fine-Grained Bioacoustics Case Study](https://mlsanity.com/paper/2605.14031/) (2026)
- [BioDCASE: Active Learning for Bioacoustics](https://mlsanity.com/paper/2609.15255/) (2026)
- [SounDiT: Geo-Contextual Soundscape-to-Landscape Generation](https://mlsanity.com/paper/2505.12734/) (2025)
- [Adversarial Training Improves Generalization Under Distribution Shifts in Bird Sound Classification](https://mlsanity.com/paper/2507.13727/) (2025)
- [Compact Hypercube Embeddings for Fast Text-based Wildlife Observation Retrieval](https://mlsanity.com/paper/2601.22783/) (2026)
- [Zwitscherkasten -- DIY Audiovisual bird monitoring](https://mlsanity.com/paper/2602.13330/) (2026)
- [BAT: Better Audio Transformer Guided by Convex Gated Probing](https://mlsanity.com/paper/2602.16305/) (2026)
- [Ecologically-Constrained Task Arithmetic for Multi-Taxa Bioacoustic Classifiers Without Shared Data](https://mlsanity.com/paper/2605.03914/) (2026)
- [A strongly annotated passive acoustic dataset for tropical bird monitoring](https://mlsanity.com/paper/2605.20578/) (2026)
- [Time-frequency localization of bird calls in dense soundscapes](https://mlsanity.com/paper/2606.10407/) (2026)
- [Adaptive Loss Balancing for Multi-Task Bioacoustic Classification of Bird Species and Call Types](https://mlsanity.com/paper/2607.03304/) (2026)
- [Can Tokens Compete? Token Representations against Supervised CNN Backbones for BirdCLEF+ 2026](https://mlsanity.com/paper/2607.14474/) (2026)

## Abstract (arXiv)

Deep learning (DL) has greatly advanced audio classification, yet the field is limited by the scarcity of large-scale benchmark datasets that have propelled progress in other domains. While AudioSet is a pivotal step to bridge this gap as a universal-domain dataset, its restricted accessibility and limited range of evaluation use cases challenge its role as the sole resource. Therefore, we introduce BirdSet, a large-scale benchmark dataset for audio classification focusing on avian bioacoustics. BirdSet surpasses AudioSet with over 6,800 recording hours (uparrow!17%) from nearly 10,000 classes (uparrow!18×) for training and more than 400 hours (uparrow!7×) across eight strongly labeled evaluation datasets. It serves as a versatile resource for use cases such as multi-label classification, covariate shift or self-supervised learning. We benchmark six well-known DL models in multi-label classification across three distinct training scenarios and outline further evaluation use cases in audio classification. We host our dataset on Hugging Face for easy accessibility and offer an extensive codebase to reproduce our results.