mlsanityData to 19 Oct 2026

Neural Representations, Natural Connections: What Transfers From Human Speech Foundation Models to Animal Vocalizations?

arXiv 2610.07107 ↗First posted 5 Oct 2026cs.SD eess.AScited by 0

On the map: Speech and audio language models → Vocoders and speech synthesis also fits: Audio language models and spoken dialogue

Abstract (arXiv)

Speech-pretrained models have shown promise in animal bioacoustics, but the factors governing their cross-species transfer remain poorly understood. We evaluate 15 frozen encoders spanning monolingual and multilingual speech, speaker verification, animal bioacoustics, and general audio on caller identification across four species and call-type classification across three. Differences between the strongest speech- and animal-pretrained representations range from -0.027 to +0.119 UAR; speech is significantly better in three of seven settings (p<0.01) and not significantly different in the remaining four. Controlled comparisons show no systematic advantage from increased language coverage or animal-domain pretraining, while frozen speaker-verification embeddings transfer poorly. The results also show that transfer is strongly layer-dependent: raw-waveform models peak early, whereas patch-spectrogram models peak deeper.

Key facts. No citing papers in the map yet (data through 19 Oct 2026). Placed in Vocoders and speech synthesis, one of 11 directions in Speech and audio language models. Builds on 14 of its 22 references that are in the map. First posted on arXiv 5 Oct 2026.

Builds on

14 of 22 references.

References · 14

  1. 2022Robust Speech Recognition via Large-Scale Weak Supervisioncited by 4,661
  2. 2020wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representationscited by 4,385
  3. 2021HuBERT: Self-Supervised Speech Representation Learning by Masked Prediction of Hidden Unitscited by 2,762
  4. 2021WavLM: Large-Scale Self-Supervised Pre-Training for Full Stack Speech Processingcited by 2,135
  5. 2021XLS-R: Self-supervised Cross-lingual Speech Representation Learning at Scalecited by 678
  6. 2022BEATs: Audio Pre-Training with Acoustic Tokenizerscited by 422
  7. 2021Layer-wise Analysis of a Self-supervised Speech Representation Modelcited by 345
  8. 2024mHuBERT-147: A Compact Multilingual HuBERT Modelcited by 73
  9. 2021UniSpeech-SAT: Universal Speech Representation Learning with Speaker Aware Pre-Trainingcited by 69
  10. 2024Towards Robust Speech Representation Learning for Thousands of Languagescited by 61
  11. 2022AVES: Animal Vocalization Encoder based on Self-Supervisioncited by 54
  12. 2023Can Self-Supervised Neural Representations Pre-Trained on Human Speech distinguish Animal Callers?cited by 11
2 more
  1. 2025Comparing Self-Supervised Learning Models Pre-Trained on Human Speech and Animal Vocalizations for Bioacoustics Processingcited by 10
  2. 2025Crossing the Species Divide: Transfer Learning from Speech to Animal Soundscited by 5

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2024animal2vec and MeerKAT: A self-supervised transformer for rare-event raw audio input and a large-scale reference dataset for bioacoustics
  2. 2025Towards Leveraging Sequential Structure in Animal Vocalizations
  3. 2026BEANS-Next and ROOTS: Broadening Audio-Language Capabilities for Bioacoustics
  4. 2025Perch 2.0: The Bittern Lesson for Bioacoustics
  5. 2024Biodenoising: Animal Vocalization Denoising without Access to Clean Data
  6. 2024Generalization in birdsong classification: impact of transfer learning methods and dataset characteristics
  7. 2026Can Tokens Compete? Token Representations against Supervised CNN Backbones for BirdCLEF+ 2026
  8. 2024Towards Dog Bark Decoding: Leveraging Human Speech Processing for Automated Bark Classification