mlsanityData to 7 Oct 2026

On the Utility of Speech and Audio Foundation Models for Marmoset Call Analysis

arXiv 2407.16417 ↗First posted 23 Jul 2024cs.SD cs.LG eess.AScited by 4

Not on the map. Why?

Abstract (arXiv)

Marmoset monkeys encode vital information in their calls and serve as a surrogate model for neuro-biologists to understand the evolutionary origins of human vocal communication. Traditionally analyzed with signal processing-based features, recent approaches have utilized self-supervised models pre-trained on human speech for feature extraction, capitalizing on their ability to learn a signal's intrinsic structure independently of its acoustic domain. However, the utility of such foundation models remains unclear for marmoset call analysis in terms of multi-class classification, bandwidth, and pre-training domain. This study assesses feature representations derived from speech and general audio domains, across pre-training bandwidths of 4, 8, and 16 kHz for marmoset call-type and caller classification tasks. Results show that models with higher bandwidth improve performance, and pre-training on speech or general audio yields comparable results, improving over a spectral baseline.

Key facts. Cited by 4 papers in the map (data through 7 Oct 2026). Builds on 9 of its 33 references that are in the map. First posted on arXiv 23 Jul 2024.

Builds on

9 of 33 references.

References · 9

  1. 2021WavLM: Large-Scale Self-Supervised Pre-Training for Full Stack Speech Processingcited by 1,609
  2. 2021SUPERB: Speech processing Universal PERformance Benchmarkcited by 652
  3. 2019PANNs: Large-Scale Pretrained Audio Neural Networks for Audio Pattern Recognitioncited by 563
  4. 2020Contrastive Learning of General-Purpose Audio Representationscited by 168
  5. 2021BYOL for Audio: Self-Supervised Learning for General-Purpose Audio Representationcited by 106
  6. 2019catch22: CAnonical Time-series CHaracteristicscited by 103
  7. 2021Computational bioacoustics with deep learning: a review and roadmapcited by 78
  8. 2013Highly comparative time-series analysis: The empirical structure of time series and their methodscited by 30
  9. 2023Can Self-Supervised Neural Representations Pre-Trained on Human Speech distinguish Animal Callers?cited by 9

Built on by

Cited by year

Citing papers per year: 2025: 4.

4 papers.

  1. 2025Comparing Self-Supervised Learning Models Pre-Trained on Human Speech and Animal Vocalizations for Bioacoustics Processingcited by 9
  2. 2025Crossing the Species Divide: Transfer Learning from Speech to Animal Soundscited by 4
  3. 2025Towards Leveraging Sequential Structure in Animal Vocalizationscited by 1
  4. 2025An Entropy-Guided Curriculum Learning Strategy for Data-Efficient Acoustic Scene Classification under Domain Shiftcited by 0