mlsanityData to 7 Oct 2026

Perch 2.0: The Bittern Lesson for Bioacoustics

arXiv 2508.04665 ↗First posted 6 Aug 2025cs.LG cs.SD eess.AScited by 25

Not on the map. Why?

Abstract (arXiv)

Perch is a performant pre-trained model for bioacoustics. It was trained in supervised fashion, providing both off-the-shelf classification scores for thousands of vocalizing species as well as strong embeddings for transfer learning. In this new release, Perch 2.0, we expand from training exclusively on avian species to a large multi-taxa dataset. The model is trained with self-distillation using a prototype-learning classifier as well as a new source-prediction training criterion. Perch 2.0 obtains state-of-the-art performance on the BirdSet and BEANS benchmarks. It also outperforms specialized marine models on marine transfer learning tasks, despite having almost no marine training data. We present hypotheses as to why fine-grained species classification is a particularly robust pre-training task for bioacoustics.

Key facts. Cited by 25 papers in the map (data through 7 Oct 2026). Builds on 32 of its 59 references that are in the map. First posted on arXiv 6 Aug 2025.

Builds on

32 of 59 references.

References · 32

  1. 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 11,979
  2. 2023DINOv2: Learning Robust Visual Features without Supervisioncited by 7,764
  3. 2019EfficientNet: Rethinking Model Scaling for Convolutional Neural Networkscited by 6,415
  4. 2020wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representationscited by 3,853
  5. 2020Big Self-Supervised Models are Strong Semi-Supervised Learnerscited by 1,375
  6. 2018This Looks Like That: Deep Learning for Interpretable Image Recognitioncited by 731
  7. 2018Exploring the Limits of Weakly Supervised Pretrainingcited by 604
  8. 2020FSD50K: An Open Dataset of Human-Labeled Sound Eventscited by 339
  9. 2020Towards Understanding Ensemble, Knowledge Distillation and Self-Distillation in Deep Learningcited by 292
  10. 2023A Survey on Self-supervised Learning: Algorithms, Applications, and Future Trendscited by 205
  11. 2020Contrastive Learning of General-Purpose Audio Representationscited by 168
  12. 2021Deformable ProtoPNet: An Interpretable Image Classifier Using Deformable Prototypescited by 120
20 more
  1. 2021When Does Contrastive Visual Representation Learning Work?cited by 92
  2. 2021Computational bioacoustics with deep learning: a review and roadmapcited by 78
  3. 2022HEAR: Holistic Evaluation of Audio Representationscited by 77
  4. 2022AVES: Animal Vocalization Encoder based on Self-Supervisioncited by 53
  5. 2023Global birdsong embeddings enable superior transfer learning for bioacoustic classificationcited by 51
  6. 2022BEANS: The Benchmark of Animal Soundscited by 37
  7. 2024BirdSet: A Large-Scale Dataset for Audio Classification in Avian Bioacousticscited by 36
  8. 2020CLAR: Contrastive Learning of Auditory Representationscited by 31
  9. 2025Can Masked Autoencoders Also Listen to Birds?cited by 26
  10. 2021Improving Bird Classification with Unsupervised Sound Separationcited by 25
  11. 2024Specialized Foundation Models Struggle to Beat Supervised Baselinescited by 23
  12. 2023BIRB: A Generalization Benchmark for Information Retrieval in Bioacousticscited by 19
  13. 2025The iNaturalist Sounds Datasetcited by 17
  14. 2023In Search for a Generalizable Method for Source Free Domain Adaptationcited by 12
  15. 2024AudioProtoPNet: An interpretable deep learning model for bird sound classificationcited by 10
  16. 2024Augmentations vs Algorithms: What Works in Self-Supervised Learningcited by 9
  17. 2024Domain-Invariant Representation Learning of Bird Soundscited by 9
  18. 2025Clustering and novel class recognition: evaluating bioacoustic deep learning feature extractorscited by 6
  19. 2023Unsupervised Learning on a DIET: Datum IndEx as Target Free of Self-Supervision, Reconstruction, Projector Headcited by 3
  20. 2024Why Fine-grained Labels in Pretraining Benefit Generalization?cited by 3

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 3; 2026: 22.

25 papers.

Sort:

  1. 2025Perch 2.0 transfers 'whale' to underwater taskscited by 7
  2. 2026Future of Edge AI in biodiversity monitoringcited by 3
  3. 2026Finding Needles in the Haystack: Transductive Active Labeling in Ecologycited by 3
  4. 2025Model Merging Improves Zero-Shot Generalization in Bioacoustic Foundation Modelscited by 2
  5. 2026Linear probing enables Ship-Radiated Noise recognition with pretrained audio embeddingscited by 2
  6. 2026Determinantal point process sampling for bioacoustic active learningcited by 2
  7. 2026MetaPerch: Learning from metadata for bioacoustics foundation modelscited by 2
  8. 2025Uncertainty Calibration of Multi-Label Bird Sound Classifierscited by 1
  9. 2026bacpipe: a Python package to make bioacoustic deep learning models accessiblecited by 1
  10. 2026Multi-layer attentive probing improves transfer of audio representations for bioacousticscited by 1
  11. 2026Masked Autoencoders with Limited Data: Does It Work? A Fine-Grained Bioacoustics Case Studycited by 1
  12. 2026BioDCASE: Active Learning for Bioacousticscited by 1
13 more
  1. 2026Adaptive Evidence Weighting for Audio-Spatiotemporal Fusioncited by 0
  2. 2026DeepForestSound: a multi-species automatic detector for passive acoustic monitoring in African tropical forests, a case study in Kibale National Parkcited by 0
  3. 2026Beyond the Baseband: Adaptive Multi-Band Encoding for Full-Spectrum Bioacoustics Classificationcited by 0
  4. 2026Ecologically-Constrained Task Arithmetic for Multi-Taxa Bioacoustic Classifiers Without Shared Datacited by 0
  5. 2026Time-frequency localization of bird calls in dense soundscapescited by 0
  6. 2026Decoding Insect Song: A Multitask Semisupervised Orthoptera Bioacoustic Classifiercited by 0
  7. 2026Can Tokens Compete? Token Representations against Supervised CNN Backbones for BirdCLEF+ 2026cited by 0
  8. 2026A Parameter-Free Few-Shot Evaluation for Elephant Vocalisation Classificationcited by 0
  9. 2026Efficient Passive Acoustic Monitoring of Killer Whales Using a Two-Stage Detection and Ecotype Classification Cascadecited by 0
  10. 2026Unlabeled Echoes: Pseudo-Labels and Genus-Aware Smoothing for Bat Call Recognitioncited by 0
  11. 2026MAST: Label-Efficient, Robust, and Generalizable Sound Detection for Biodiversity Monitoring via Masked Audio Pretraining and Self-Trainingcited by 0
  12. 2026OpenWhistle: A Large-Scale Longitudinal Dataset and Benchmark of Bottlenose Dolphin Vocalizationscited by 0
  13. 2026A Strength-Monotonic Law for Domain Alignment in Frozen-Embedding Bioacoustic Classificationcited by 0