mlsanityData to 7 Oct 2026

Exploring Speech Foundation Models for Speaker Diarization in Child-Adult Dyadic Interactions

arXiv 2406.07890 ↗First posted 12 Jun 2024eess.AS cs.CL cs.LGcited by 4

Not on the map. Why?

Abstract (arXiv)

Speech foundation models, trained on vast datasets, have opened unique opportunities in addressing challenging low-resource speech understanding, such as child speech. In this work, we explore the capabilities of speech foundation models on child-adult speaker diarization. We show that exemplary foundation models can achieve 39.5% and 62.3% relative reductions in Diarization Error Rate and Speaker Confusion Rate, respectively, compared to previous speaker diarization methods. In addition, we benchmark and evaluate the speaker diarization results of the speech foundation models with varying the input audio window size, speaker demographics, and training data ratio. Our results highlight promising pathways for understanding and adopting speech foundation models to facilitate child speech understanding.

Key facts. Cited by 4 papers in the map (data through 7 Oct 2026). Builds on 19 of its 33 references that are in the map. First posted on arXiv 12 Jun 2024.

Builds on

19 of 33 references.

References · 19

  1. 2022Robust Speech Recognition via Large-Scale Weak Supervisioncited by 3,972
  2. 2021On the Opportunities and Risks of Foundation Modelscited by 3,961
  3. 2020wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representationscited by 3,853
  4. 2021HuBERT: Self-Supervised Speech Representation Learning by Masked Prediction of Hidden Unitscited by 2,301
  5. 2019HuggingFace's Transformers: State-of-the-art Natural Language Processingcited by 2,201
  6. 2021WavLM: Large-Scale Self-Supervised Pre-Training for Full Stack Speech Processingcited by 1,609
  7. 2017VoxCeleb: a large-scale speaker identification datasetcited by 714
  8. 2020ECAPA-TDNN: Emphasized Channel Attention, Propagation and Aggregation in TDNN Based Speaker Verificationcited by 606
  9. 2023Scaling Speech Technology to 1,000+ Languagescited by 401
  10. 2023Powerset multi-class cross entropy loss for neural speaker diarizationcited by 109
  11. 2019End-to-End Neural Speaker Diarization with Permutation-Free Objectivescited by 86
  12. 2020Bayesian HMM clustering of x-vector sequences (VBx) in speaker diarization: theory, implementation and analysis on standard taskscited by 69
7 more
  1. 2019CN-CELEB: a challenging Chinese speaker recognition datasetcited by 47
  2. 2020Integrating end-to-end neural and clustering-based diarization: Getting the best of both worldscited by 33
  3. 2021Advances in integration of end-to-end neural and clustering-based diarization for real conversational speechcited by 22
  4. 2023Towards Robust Family-Infant Audio Analysis Based on Unsupervised Pretraining of Wav2vec 2.0 on Large-Scale Unlabeled Family Audiocited by 6
  5. 2023Robust Self Supervised Speech Embeddings for Child-Adult Classification in Interactions involving Children with Autismcited by 4
  6. 2023Understanding Spoken Language Development of Children with ASD Using Pre-trained Speech Embeddingscited by 3
  7. 2023Audio-visual child-adult speaker classification in dyadic interactionscited by 2

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 1; 2025: 2; 2026: 1.

4 papers.

  1. 2024Can Generic LLMs Help Analyze Child-adult Interactions Involving Children with Autism in Clinical Observation?cited by 2
  2. 2025Challenges in Automated Processing of Speech from Child Wearables: The Case of Voice Type Classifiercited by 2
  3. 2025K-Function: Joint Pronunciation Transcription and Feedback for Evaluating Kids Language Functioncited by 1
  4. 2026Robust Multi-Tier Infant-Centered Audio Understanding with Whisper via Structured Speaker Conditioningcited by 0