# AudioProtoPNet: An interpretable deep learning model for bird sound classification

arXiv: https://arxiv.org/abs/2404.10420 · HTML: https://mlsanity.com/paper/2404.10420/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- Builds on 32 of its 74 references that are in the map.
- First posted on arXiv 16 Apr 2024.

## Builds on (most cited first, 30 of 32)

- [EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks](https://mlsanity.com/paper/1905.11946/) (2019)
- [mixup: Beyond Empirical Risk Minimization](https://mlsanity.com/paper/1710.09412/) (2017)
- ["Why Should I Trust You?": Explaining the Predictions of Any Classifier](https://mlsanity.com/paper/1602.04938/) (2016)
- [SGDR: Stochastic Gradient Descent with Warm Restarts](https://mlsanity.com/paper/1608.03983/) (2016)
- [A ConvNet for the 2020s](https://mlsanity.com/paper/2201.03545/) (2022)
- [Shortcut Learning in Deep Neural Networks](https://mlsanity.com/paper/2004.07780/) (2020)
- [Explainable Artificial Intelligence (XAI): Concepts, Taxonomies, Opportunities and Challenges toward Responsible AI](https://mlsanity.com/paper/1910.10045/) (2019)
- [SpecAugment: A Simple Data Augmentation Method for Automatic Speech Recognition](https://mlsanity.com/paper/1904.08779/) (2019)
- [AugMix: A Simple Data Processing Method to Improve Robustness and Uncertainty](https://mlsanity.com/paper/1912.02781/) (2019)
- [This Looks Like That: Deep Learning for Interpretable Image Recognition](https://mlsanity.com/paper/1806.10574/) (2018)
- [Asymmetric Loss For Multi-Label Classification](https://mlsanity.com/paper/2009.14119/) (2020)
- [Interpretable Machine Learning: Fundamental Principles and 10 Grand Challenges](https://mlsanity.com/paper/2103.11251/) (2021)
- [Masked Autoencoders that Listen](https://mlsanity.com/paper/2207.06405/) (2022)
- [Deep Learning for Case-Based Reasoning through Prototypes: A Neural Network that Explains Its Predictions](https://mlsanity.com/paper/1710.04806/) (2017)
- [Neural Prototype Trees for Interpretable Fine-grained Image Recognition](https://mlsanity.com/paper/2012.02046/) (2020)
- [Deformable ProtoPNet: An Interpretable Image Classifier Using Deformable Prototypes](https://mlsanity.com/paper/2111.15000/) (2021)
- [Interpretable Image Classification with Differentiable Prototypes Assignment](https://mlsanity.com/paper/2112.02902/) (2021)
- [TorchAudio: Building Blocks for Audio and Speech Processing](https://mlsanity.com/paper/2110.15018/) (2021)
- [Computational bioacoustics with deep learning: a review and roadmap](https://mlsanity.com/paper/2112.06725/) (2021)
- [ProtoPFormer: Concentrating on Prototypical Parts in Vision Transformers for Interpretable Image Recognition](https://mlsanity.com/paper/2208.10431/) (2022)
- [Global birdsong embeddings enable superior transfer learning for bioacoustic classification](https://mlsanity.com/paper/2307.06292/) (2023)
- [This Looks Like That, Because ... Explaining Prototypes for Interpretable Image Recognition](https://mlsanity.com/paper/2011.02863/) (2020)
- [Automatic acoustic detection of birds through deep learning: the first Bird Audio Detection challenge](https://mlsanity.com/paper/1807.05812/) (2018)
- [This Looks Like Those: Illuminating Prototypical Concepts Using Multiple Visualizations](https://mlsanity.com/paper/2310.18589/) (2023)
- [Toward Interpretable Music Tagging with Self-Attention](https://mlsanity.com/paper/1906.04972/) (2019)
- [Towards Relatable Explainable AI with the Perceptual Process](https://mlsanity.com/paper/2112.14005/) (2021)
- [Improving Bird Classification with Unsupervised Sound Separation](https://mlsanity.com/paper/2110.03209/) (2021)
- [Prototype Learning for Interpretable Respiratory Sound Analysis](https://mlsanity.com/paper/2110.03536/) (2021)
- [BIRB: A Generalization Benchmark for Information Retrieval in Bioacoustics](https://mlsanity.com/paper/2312.07439/) (2023)
- [NIPS4Bplus: a richly annotated birdsong audio dataset](https://mlsanity.com/paper/1811.02275/) (2018)

## Built on by (most cited first, 10 of 10)

- [Can Masked Autoencoders Also Listen to Birds?](https://mlsanity.com/paper/2504.12880/) (2025)
- [Perch 2.0: The Bittern Lesson for Bioacoustics](https://mlsanity.com/paper/2508.04665/) (2025)
- [Foundation Models for Bioacoustics -- a Comparative Review](https://mlsanity.com/paper/2508.01277/) (2025)
- [Unmute the Patch Tokens: Rethinking Probing in Multi-Label Audio Classification](https://mlsanity.com/paper/2509.24901/) (2025)
- [bacpipe: a Python package to make bioacoustic deep learning models accessible](https://mlsanity.com/paper/2604.11560/) (2026)
- [MetaPerch: Learning from metadata for bioacoustics foundation models](https://mlsanity.com/paper/2607.14072/) (2026)
- [Explainability of CNN Based Classification Models for Acoustic Signal](https://mlsanity.com/paper/2509.08717/) (2025)
- [Uncertainty Calibration of Multi-Label Bird Sound Classifiers](https://mlsanity.com/paper/2511.08261/) (2025)
- [Adversarial Training Improves Generalization Under Distribution Shifts in Bird Sound Classification](https://mlsanity.com/paper/2507.13727/) (2025)
- [Spectrogram features for audio and speech analysis](https://mlsanity.com/paper/2603.14917/) (2026)

## Abstract (arXiv)

Deep learning models have significantly advanced acoustic bird monitoring by being able to recognize numerous bird species based on their vocalizations. However, traditional deep learning models are black boxes that provide no insight into their underlying computations, limiting their usefulness to ornithologists and machine learning engineers. Explainable models could facilitate debugging, knowledge discovery, trust, and interdisciplinary collaboration. This study introduces AudioProtoPNet, an adaptation of the Prototypical Part Network (ProtoPNet) for multi-label bird sound classification. It is an inherently interpretable model that uses a ConvNeXt backbone to extract embeddings, with the classification layer replaced by a prototype learning classifier trained on these embeddings. The classifier learns prototypical patterns of each bird species' vocalizations from spectrograms of training instances. During inference, audio recordings are classified by comparing them to the learned prototypes in the embedding space, providing explanations for the model's decisions and insights into the most informative embeddings of each bird species. The model was trained on the BirdSet training dataset, which consists of 9,734 bird species and over 6,800 hours of recordings. Its performance was evaluated on the seven test datasets of BirdSet, covering different geographical regions. AudioProtoPNet outperformed the state-of-the-art model Perch, achieving an average AUROC of 0.90 and a cmAP of 0.42, with relative improvements of 7.1% and 16.7% over Perch, respectively. These results demonstrate that even for the challenging task of multi-label bird sound classification, it is possible to develop powerful yet inherently interpretable deep learning models that provide valuable insights for ornithologists and machine learning engineers.