# Perch 2.0: The Bittern Lesson for Bioacoustics

arXiv: https://arxiv.org/abs/2508.04665 · HTML: https://mlsanity.com/paper/2508.04665/

## Key facts

- Cited by 26 papers in the map (data through 14 Oct 2026).
- Builds on 32 of its 59 references that are in the map.
- First posted on arXiv 6 Aug 2025.

## Builds on (most cited first, 30 of 32)

- [A Simple Framework for Contrastive Learning of Visual Representations](https://mlsanity.com/paper/2002.05709/) (2020)
- [DINOv2: Learning Robust Visual Features without Supervision](https://mlsanity.com/paper/2304.07193/) (2023)
- [EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks](https://mlsanity.com/paper/1905.11946/) (2019)
- [wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representations](https://mlsanity.com/paper/2006.11477/) (2020)
- [Big Self-Supervised Models are Strong Semi-Supervised Learners](https://mlsanity.com/paper/2006.10029/) (2020)
- [This Looks Like That: Deep Learning for Interpretable Image Recognition](https://mlsanity.com/paper/1806.10574/) (2018)
- [Exploring the Limits of Weakly Supervised Pretraining](https://mlsanity.com/paper/1805.00932/) (2018)
- [FSD50K: An Open Dataset of Human-Labeled Sound Events](https://mlsanity.com/paper/2010.00475/) (2020)
- [Towards Understanding Ensemble, Knowledge Distillation and Self-Distillation in Deep Learning](https://mlsanity.com/paper/2012.09816/) (2020)
- [A Survey on Self-supervised Learning: Algorithms, Applications, and Future Trends](https://mlsanity.com/paper/2301.05712/) (2023)
- [Contrastive Learning of General-Purpose Audio Representations](https://mlsanity.com/paper/2010.10915/) (2020)
- [Deformable ProtoPNet: An Interpretable Image Classifier Using Deformable Prototypes](https://mlsanity.com/paper/2111.15000/) (2021)
- [When Does Contrastive Visual Representation Learning Work?](https://mlsanity.com/paper/2105.05837/) (2021)
- [Computational bioacoustics with deep learning: a review and roadmap](https://mlsanity.com/paper/2112.06725/) (2021)
- [HEAR: Holistic Evaluation of Audio Representations](https://mlsanity.com/paper/2203.03022/) (2022)
- [AVES: Animal Vocalization Encoder based on Self-Supervision](https://mlsanity.com/paper/2210.14493/) (2022)
- [Global birdsong embeddings enable superior transfer learning for bioacoustic classification](https://mlsanity.com/paper/2307.06292/) (2023)
- [BEANS: The Benchmark of Animal Sounds](https://mlsanity.com/paper/2210.12300/) (2022)
- [BirdSet: A Large-Scale Dataset for Audio Classification in Avian Bioacoustics](https://mlsanity.com/paper/2403.10380/) (2024)
- [CLAR: Contrastive Learning of Auditory Representations](https://mlsanity.com/paper/2010.09542/) (2020)
- [Can Masked Autoencoders Also Listen to Birds?](https://mlsanity.com/paper/2504.12880/) (2025)
- [Improving Bird Classification with Unsupervised Sound Separation](https://mlsanity.com/paper/2110.03209/) (2021)
- [Specialized Foundation Models Struggle to Beat Supervised Baselines](https://mlsanity.com/paper/2411.02796/) (2024)
- [BIRB: A Generalization Benchmark for Information Retrieval in Bioacoustics](https://mlsanity.com/paper/2312.07439/) (2023)
- [The iNaturalist Sounds Dataset](https://mlsanity.com/paper/2506.00343/) (2025)
- [In Search for a Generalizable Method for Source Free Domain Adaptation](https://mlsanity.com/paper/2302.06658/) (2023)
- [AudioProtoPNet: An interpretable deep learning model for bird sound classification](https://mlsanity.com/paper/2404.10420/) (2024)
- [Augmentations vs Algorithms: What Works in Self-Supervised Learning](https://mlsanity.com/paper/2403.05726/) (2024)
- [Domain-Invariant Representation Learning of Bird Sounds](https://mlsanity.com/paper/2409.08589/) (2024)
- [Clustering and novel class recognition: evaluating bioacoustic deep learning feature extractors](https://mlsanity.com/paper/2504.06710/) (2025)

## Built on by (most cited first, 26 of 26)

- [Perch 2.0 transfers 'whale' to underwater tasks](https://mlsanity.com/paper/2512.03219/) (2025)
- [Future of Edge AI in biodiversity monitoring](https://mlsanity.com/paper/2602.13496/) (2026)
- [Finding Needles in the Haystack: Transductive Active Labeling in Ecology](https://mlsanity.com/paper/2606.03821/) (2026)
- [Model Merging Improves Zero-Shot Generalization in Bioacoustic Foundation Models](https://mlsanity.com/paper/2511.05171/) (2025)
- [Linear probing enables Ship-Radiated Noise recognition with pretrained audio embeddings](https://mlsanity.com/paper/2601.08358/) (2026)
- [bacpipe: a Python package to make bioacoustic deep learning models accessible](https://mlsanity.com/paper/2604.11560/) (2026)
- [Determinantal point process sampling for bioacoustic active learning](https://mlsanity.com/paper/2607.06063/) (2026)
- [MetaPerch: Learning from metadata for bioacoustics foundation models](https://mlsanity.com/paper/2607.14072/) (2026)
- [Uncertainty Calibration of Multi-Label Bird Sound Classifiers](https://mlsanity.com/paper/2511.08261/) (2025)
- [Multi-layer attentive probing improves transfer of audio representations for bioacoustics](https://mlsanity.com/paper/2605.10494/) (2026)
- [Masked Autoencoders with Limited Data: Does It Work? A Fine-Grained Bioacoustics Case Study](https://mlsanity.com/paper/2605.14031/) (2026)
- [BioDCASE: Active Learning for Bioacoustics](https://mlsanity.com/paper/2609.15255/) (2026)
- [Adaptive Evidence Weighting for Audio-Spatiotemporal Fusion](https://mlsanity.com/paper/2602.03817/) (2026)
- [DeepForestSound: a multi-species automatic detector for passive acoustic monitoring in African tropical forests, a case study in Kibale National Park](https://mlsanity.com/paper/2604.08087/) (2026)
- [Beyond the Baseband: Adaptive Multi-Band Encoding for Full-Spectrum Bioacoustics Classification](https://mlsanity.com/paper/2604.27936/) (2026)
- [Ecologically-Constrained Task Arithmetic for Multi-Taxa Bioacoustic Classifiers Without Shared Data](https://mlsanity.com/paper/2605.03914/) (2026)
- [Time-frequency localization of bird calls in dense soundscapes](https://mlsanity.com/paper/2606.10407/) (2026)
- [Decoding Insect Song: A Multitask Semisupervised Orthoptera Bioacoustic Classifier](https://mlsanity.com/paper/2606.13236/) (2026)
- [Can Tokens Compete? Token Representations against Supervised CNN Backbones for BirdCLEF+ 2026](https://mlsanity.com/paper/2607.14474/) (2026)
- [A Parameter-Free Few-Shot Evaluation for Elephant Vocalisation Classification](https://mlsanity.com/paper/2608.14824/) (2026)
- [ChiroEcho: extending automated bat vocalisation classification beyond the learned taxonomy](https://mlsanity.com/paper/2608.18191/) (2026)
- [Efficient Passive Acoustic Monitoring of Killer Whales Using a Two-Stage Detection and Ecotype Classification Cascade](https://mlsanity.com/paper/2609.01792/) (2026)
- [Unlabeled Echoes: Pseudo-Labels and Genus-Aware Smoothing for Bat Call Recognition](https://mlsanity.com/paper/2609.11986/) (2026)
- [MAST: Label-Efficient, Robust, and Generalizable Sound Detection for Biodiversity Monitoring via Masked Audio Pretraining and Self-Training](https://mlsanity.com/paper/2609.15221/) (2026)
- [OpenWhistle: A Large-Scale Longitudinal Dataset and Benchmark of Bottlenose Dolphin Vocalizations](https://mlsanity.com/paper/2609.34839/) (2026)
- [A Strength-Monotonic Law for Domain Alignment in Frozen-Embedding Bioacoustic Classification](https://mlsanity.com/paper/2610.09737/) (2026)

## Abstract (arXiv)

Perch is a performant pre-trained model for bioacoustics. It was trained in supervised fashion, providing both off-the-shelf classification scores for thousands of vocalizing species as well as strong embeddings for transfer learning. In this new release, Perch 2.0, we expand from training exclusively on avian species to a large multi-taxa dataset. The model is trained with self-distillation using a prototype-learning classifier as well as a new source-prediction training criterion. Perch 2.0 obtains state-of-the-art performance on the BirdSet and BEANS benchmarks. It also outperforms specialized marine models on marine transfer learning tasks, despite having almost no marine training data. We present hypotheses as to why fine-grained species classification is a particularly robust pre-training task for bioacoustics.