mlsanityData to 13 Oct 2026

Benchmarking Children's ASR with Supervised and Self-supervised Speech Foundation Models

arXiv 2406.10507 ↗First posted 15 Jun 2024eess.AS cs.CL cs.SDcited by 29

Not on the map. Why?

Abstract (arXiv)

Speech foundation models (SFMs) have achieved state-of-the-art results for various speech tasks in supervised (e.g. Whisper) or self-supervised systems (e.g. WavLM). However, the performance of SFMs for child ASR has not been systematically studied. In addition, there is no benchmark for child ASR with standard evaluations, making the comparisons of novel ideas difficult. In this paper, we initiate and present a comprehensive benchmark on several child speech databases based on various SFMs (Whisper, Wav2vec2.0, HuBERT, and WavLM). Moreover, we investigate finetuning strategies by comparing various data augmentation and parameter-efficient finetuning (PEFT) methods. We observe that the behaviors of these methods are different when the model size increases. For example, PEFT matches the performance of full finetuning for large models but worse for small models. To stabilize finetuning using augmented data, we propose a perturbation invariant finetuning (PIF) loss as a regularization.

Key facts. Cited by 29 papers in the map (data through 13 Oct 2026). Builds on 27 of its 37 references that are in the map. First posted on arXiv 15 Jun 2024.

Builds on

27 of 37 references.

References · 27

  1. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,231
  2. 2019Parameter-Efficient Transfer Learning for NLPcited by 4,213
  3. 2022Robust Speech Recognition via Large-Scale Weak Supervisioncited by 4,004
  4. 2021Prefix-Tuning: Optimizing Continuous Prompts for Generationcited by 3,898
  5. 2020wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representationscited by 3,870
  6. 2021HuBERT: Self-Supervised Speech Representation Learning by Masked Prediction of Hidden Unitscited by 2,313
  7. 2021WavLM: Large-Scale Self-Supervised Pre-Training for Full Stack Speech Processingcited by 1,625
  8. 2019SpecAugment: A Simple Data Augmentation Method for Automatic Speech Recognitioncited by 1,359
  9. 2023LIMA: Less Is More for Alignmentcited by 1,091
  10. 2023Voicebox: Text-Guided Multilingual Universal Speech Generation at Scalecited by 376
  11. 2021W2v-BERT: Combining Contrastive Learning and Masked Language Modeling for Self-Supervised Speech Pre-Trainingcited by 342
  12. 2022Self-Supervised Speech Representation Learning: A Reviewcited by 258
15 more
  1. 2019NeMo: a toolkit for building AI applications using Neural Modulescited by 232
  2. 2023Seamless: Multilingual Expressive and Streaming Speech Translationcited by 190
  3. 2023Fast Conformer with Linearly Scalable Attention for Efficient Speech Recognitioncited by 122
  4. 2021BigSSL: Exploring the Frontier of Large-Scale Semi-Supervised Learning for Automatic Speech Recognitioncited by 88
  5. 2023Prompting Large Language Models for Zero-Shot Domain Adaptation in Speech Recognitioncited by 48
  6. 2023Reproducing Whisper-Style Training Using an Open-Source Toolkit and Publicly Available Datacited by 42
  7. 2023Kid-Whisper: Towards Bridging the Performance Gap in Automatic Speech Recognition for Children VS. Adultscited by 27
  8. 2023Adaptation of Whisper models to child speech recognitioncited by 26
  9. 2023Towards Better Domain Adaptation for Self-supervised Models: A Case Study of Child ASRcited by 23
  10. 2018Transfer Learning from Adult to Children for Speech Recognition: Evaluation, Analysis and Recommendationscited by 22
  11. 2022DRAFT: A Novel Framework to Reduce Domain Shifting in Self-supervised Learning and Its Application to Children's ASRcited by 21
  12. 2023Sparsely Shared LoRA on Whisper for Child Speech Recognitioncited by 18
  13. 2021Fundamental Frequency Feature Normalization and Data Augmentation for Child Speech Recognitioncited by 9
  14. 2021Bi-APC: Bidirectional Autoregressive Predictive Coding for Unsupervised Pre-training and Its Application to Children's ASRcited by 5
  15. 2022Improving Children's Speech Recognition by Fine-tuning Self-supervised Adult Speech Representationscited by 3

Built on by

Cited by year (2026 to 13 Oct)

Citing papers per year: 2024: 5; 2025: 15; 2026: 9.

29 papers.

Sort:

By area:

  1. 2025Selective Attention Merging for low resource tasks: A case study of Child ASRcited by 6
  2. 2025Benchmarking Training Paradigms, Dataset Composition, and Model Scaling for Child ASR in ESPnetcited by 6
  3. 2025Speech Recognition for Automatically Assessing Afrikaans and isiXhosa Preschool Oral Narrativescited by 4
  4. 2025SSVD: Structured SVD for Parameter-Efficient Fine-Tuning and Benchmarking under Domain Shift in ASRcited by 4
  5. 2024Can Generic LLMs Help Analyze Child-adult Interactions Involving Children with Autism in Clinical Observation?cited by 3
  6. 2025Improving Child Speech Recognition and Reading Mistake Detection by Using Promptscited by 3
  7. 2025TICL: Text-Embedding KNN For Speech In-Context Learning Unlocks Speech Recognition Abilities of Large Multimodal Modelscited by 3
  8. 2025Finding My Voice: Generative Reconstruction of Disordered Speech for Automated Clinical Evaluationcited by 3
  9. 2026Mind the Shift: Using Delta SSL Embeddings to Enhance Child ASRcited by 3
  10. 2025Large Language Models based ASR Error Correction for Child Conversationscited by 2
  11. 2025The Sound of Syntax: Finetuning and Comprehensive Evaluation of Language Models for Speech Pathologycited by 2
  12. 2024Egocentric Speaker Classification in Child-Adult Dyadic Interactions: From Sensing to Computational Modelingcited by 1
17 more
  1. 2024Evaluation of Speech Foundation Models for ASR on Child-Adult Conversations in Autism Diagnostic Sessionscited by 1
  2. 2024Detecting Children with Autism Spectrum Disorder based on Script-Centric Behavior Understanding with Emotional Enhancementcited by 1
  3. 2025SimClass: A Classroom Speech Dataset Generated via Game Engine Simulation For Automatic Speech Recognition Researchcited by 1
  4. 2025K-Function: Joint Pronunciation Transcription and Feedback for Evaluating Kids Language Functioncited by 1
  5. 2026Entropy-Aware Domain-Routed Mixture-of-Experts Speech-LLM Framework: A Case Study of Multi-Domain Child-Adult ASRcited by 1
  6. 2026Gumbel-BEARD: Automatic Layer Selection for Self-Supervised Adaptation of Whisper in Low-Resource Domainscited by 1
  7. 2024Examining Test-Time Adaptation for Personalized Child Speech Recognitioncited by 0
  8. 2025From Weak Labels to Strong Results: Utilizing 5,000 Hours of Noisy Classroom Transcripts with Minimal Accurate Datacited by 0
  9. 2025Causal Structure Discovery for Error Diagnostics of Children's ASRcited by 0
  10. 2025Audio-3DVG: Unified Audio -- Point Cloud Fusion for 3D Visual Groundingcited by 0
  11. 2025RealClass: A Framework for Classroom Speech Simulation with Public Datasets and Game Enginescited by 0
  12. 2026Abjad-Kids: An Arabic Speech Classification Dataset for Primary Educationcited by 0
  13. 2026Child-Centric Voice Anonymization in Single and Multi-Speaker Speech via Domain-Adapted SSL Modelscited by 0
  14. 2026Encoder Awakening via Adapters: Effective Domain-Adaptive Fine-tuning of Speech-LLMscited by 0
  15. 2026AURA: Uncertainty-Routed Activation Editing for Acoustic Grounding in Speech Foundation Modelscited by 0
  16. 2026Deep Learning Techniques for Phoneme Recognition in Italian Children' s Speechcited by 0
  17. 2026Child-Adapted Structured Phonological Representations for Interpretable Speech Sound Analysiscited by 0