# Comparing Self-Supervised Learning Models Pre-Trained on Human Speech and Animal Vocalizations for Bioacoustics Processing

arXiv: https://arxiv.org/abs/2501.05987 · HTML: https://mlsanity.com/paper/2501.05987/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- Builds on 19 of its 28 references that are in the map.
- First posted on arXiv 10 Jan 2025.

## Builds on (most cited first, 19 of 19)

- [wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representations](https://mlsanity.com/paper/2006.11477/) (2020)
- [HuBERT: Self-Supervised Speech Representation Learning by Masked Prediction of Hidden Units](https://mlsanity.com/paper/2106.07447/) (2021)
- [WavLM: Large-Scale Self-Supervised Pre-Training for Full Stack Speech Processing](https://mlsanity.com/paper/2110.13900/) (2021)
- [SUPERB: Speech processing Universal PERformance Benchmark](https://mlsanity.com/paper/2105.01051/) (2021)
- [VGGSound: A Large-scale Audio-Visual Dataset](https://mlsanity.com/paper/2004.14368/) (2020)
- [FSD50K: An Open Dataset of Human-Labeled Sound Events](https://mlsanity.com/paper/2010.00475/) (2020)
- [Self-Supervised Speech Representation Learning: A Review](https://mlsanity.com/paper/2205.10643/) (2022)
- [Computational bioacoustics with deep learning: a review and roadmap](https://mlsanity.com/paper/2112.06725/) (2021)
- [AVES: Animal Vocalization Encoder based on Self-Supervision](https://mlsanity.com/paper/2210.14493/) (2022)
- [Global birdsong embeddings enable superior transfer learning for bioacoustic classification](https://mlsanity.com/paper/2307.06292/) (2023)
- [BEANS: The Benchmark of Animal Sounds](https://mlsanity.com/paper/2210.12300/) (2022)
- [Self-Supervised Learning for Few-Shot Bird Sound Classification](https://mlsanity.com/paper/2312.15824/) (2023)
- [Can Self-Supervised Neural Representations Pre-Trained on Human Speech distinguish Animal Callers?](https://mlsanity.com/paper/2305.14035/) (2023)
- [Towards Dog Bark Decoding: Leveraging Human Speech Processing for Automated Bark Classification](https://mlsanity.com/paper/2404.18739/) (2024)
- [Exploring bat song syllable representations in self-supervised audio encoders](https://mlsanity.com/paper/2409.12634/) (2024)
- [On the Utility of Speech and Audio Foundation Models for Marmoset Call Analysis](https://mlsanity.com/paper/2407.16417/) (2024)
- [Feature Representations for Automatic Meerkat Vocalization Classification](https://mlsanity.com/paper/2408.15296/) (2024)
- [On the Transferability of Large-Scale Self-Supervision to Few-Shot Audio Classification](https://mlsanity.com/paper/2402.01274/) (2024)
- [Bird Vocalization Embedding Extraction Using Self-Supervised Disentangled Representation Learning](https://mlsanity.com/paper/2412.20146/) (2024)

## Built on by (most cited first, 9 of 9)

- [AVEX: What Matters for Animal Vocalization Encoding](https://mlsanity.com/paper/2508.11845/) (2025)
- [Crossing the Species Divide: Transfer Learning from Speech to Animal Sounds](https://mlsanity.com/paper/2509.04166/) (2025)
- [MetaPerch: Learning from metadata for bioacoustics foundation models](https://mlsanity.com/paper/2607.14072/) (2026)
- [Towards Leveraging Sequential Structure in Animal Vocalizations](https://mlsanity.com/paper/2511.10190/) (2025)
- [Multi-layer attentive probing improves transfer of audio representations for bioacoustics](https://mlsanity.com/paper/2605.10494/) (2026)
- [An Entropy-Guided Curriculum Learning Strategy for Data-Efficient Acoustic Scene Classification under Domain Shift](https://mlsanity.com/paper/2509.11168/) (2025)
- [Beyond the Baseband: Adaptive Multi-Band Encoding for Full-Spectrum Bioacoustics Classification](https://mlsanity.com/paper/2604.27936/) (2026)
- [BeeVe: Unsupervised Acoustic State Discovery in Honey Bee Buzzing](https://mlsanity.com/paper/2605.07903/) (2026)
- [BioSEN: A Bio-acoustic Signal Enhancement Network for Animal Vocalizations](https://mlsanity.com/paper/2605.12534/) (2026)

## Abstract (arXiv)

Self-supervised learning (SSL) foundation models have emerged as powerful, domain-agnostic, general-purpose feature extractors applicable to a wide range of tasks. Such models pre-trained on human speech have demonstrated high transferability for bioacoustic processing. This paper investigates (i) whether SSL models pre-trained directly on animal vocalizations offer a significant advantage over those pre-trained on speech, and (ii) whether fine-tuning speech-pretrained models on automatic speech recognition (ASR) tasks can enhance bioacoustic classification. We conduct a comparative analysis using three diverse bioacoustic datasets and two different bioacoustic tasks. Results indicate that pre-training on bioacoustic data provides only marginal improvements over speech-pretrained models, with comparable performance in most scenarios. Fine-tuning on ASR tasks yields mixed outcomes, suggesting that the general-purpose representations learned during SSL pre-training are already well-suited for bioacoustic tasks. These findings highlight the robustness of speech-pretrained SSL models for bioacoustics and imply that extensive fine-tuning may not be necessary for optimal performance.