# Benchmarking Children's ASR with Supervised and Self-supervised Speech Foundation Models

arXiv: https://arxiv.org/abs/2406.10507 · HTML: https://mlsanity.com/paper/2406.10507/

## Key facts

- Cited by 29 papers in the map (data through 14 Oct 2026).
- Builds on 27 of its 37 references that are in the map.
- First posted on arXiv 15 Jun 2024.

## Builds on (most cited first, 27 of 27)

- [LoRA: Low-Rank Adaptation of Large Language Models](https://mlsanity.com/paper/2106.09685/) (2021)
- [Parameter-Efficient Transfer Learning for NLP](https://mlsanity.com/paper/1902.00751/) (2019)
- [Robust Speech Recognition via Large-Scale Weak Supervision](https://mlsanity.com/paper/2212.04356/) (2022)
- [Prefix-Tuning: Optimizing Continuous Prompts for Generation](https://mlsanity.com/paper/2101.00190/) (2021)
- [wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representations](https://mlsanity.com/paper/2006.11477/) (2020)
- [HuBERT: Self-Supervised Speech Representation Learning by Masked Prediction of Hidden Units](https://mlsanity.com/paper/2106.07447/) (2021)
- [WavLM: Large-Scale Self-Supervised Pre-Training for Full Stack Speech Processing](https://mlsanity.com/paper/2110.13900/) (2021)
- [SpecAugment: A Simple Data Augmentation Method for Automatic Speech Recognition](https://mlsanity.com/paper/1904.08779/) (2019)
- [LIMA: Less Is More for Alignment](https://mlsanity.com/paper/2305.11206/) (2023)
- [Voicebox: Text-Guided Multilingual Universal Speech Generation at Scale](https://mlsanity.com/paper/2306.15687/) (2023)
- [W2v-BERT: Combining Contrastive Learning and Masked Language Modeling for Self-Supervised Speech Pre-Training](https://mlsanity.com/paper/2108.06209/) (2021)
- [Self-Supervised Speech Representation Learning: A Review](https://mlsanity.com/paper/2205.10643/) (2022)
- [NeMo: a toolkit for building AI applications using Neural Modules](https://mlsanity.com/paper/1909.09577/) (2019)
- [Seamless: Multilingual Expressive and Streaming Speech Translation](https://mlsanity.com/paper/2312.05187/) (2023)
- [Fast Conformer with Linearly Scalable Attention for Efficient Speech Recognition](https://mlsanity.com/paper/2305.05084/) (2023)
- [BigSSL: Exploring the Frontier of Large-Scale Semi-Supervised Learning for Automatic Speech Recognition](https://mlsanity.com/paper/2109.13226/) (2021)
- [Prompting Large Language Models for Zero-Shot Domain Adaptation in Speech Recognition](https://mlsanity.com/paper/2306.16007/) (2023)
- [Reproducing Whisper-Style Training Using an Open-Source Toolkit and Publicly Available Data](https://mlsanity.com/paper/2309.13876/) (2023)
- [Kid-Whisper: Towards Bridging the Performance Gap in Automatic Speech Recognition for Children VS. Adults](https://mlsanity.com/paper/2309.07927/) (2023)
- [Adaptation of Whisper models to child speech recognition](https://mlsanity.com/paper/2307.13008/) (2023)
- [Towards Better Domain Adaptation for Self-supervised Models: A Case Study of Child ASR](https://mlsanity.com/paper/2305.00115/) (2023)
- [Transfer Learning from Adult to Children for Speech Recognition: Evaluation, Analysis and Recommendations](https://mlsanity.com/paper/1805.03322/) (2018)
- [DRAFT: A Novel Framework to Reduce Domain Shifting in Self-supervised Learning and Its Application to Children's ASR](https://mlsanity.com/paper/2206.07931/) (2022)
- [Sparsely Shared LoRA on Whisper for Child Speech Recognition](https://mlsanity.com/paper/2309.11756/) (2023)
- [Fundamental Frequency Feature Normalization and Data Augmentation for Child Speech Recognition](https://mlsanity.com/paper/2102.09106/) (2021)
- [Bi-APC: Bidirectional Autoregressive Predictive Coding for Unsupervised Pre-training and Its Application to Children's ASR](https://mlsanity.com/paper/2102.06816/) (2021)
- [Improving Children's Speech Recognition by Fine-tuning Self-supervised Adult Speech Representations](https://mlsanity.com/paper/2211.07769/) (2022)

## Built on by (most cited first, 29 of 29)

- [Selective Attention Merging for low resource tasks: A case study of Child ASR](https://mlsanity.com/paper/2501.08468/) (2025)
- [Benchmarking Training Paradigms, Dataset Composition, and Model Scaling for Child ASR in ESPnet](https://mlsanity.com/paper/2508.16576/) (2025)
- [Speech Recognition for Automatically Assessing Afrikaans and isiXhosa Preschool Oral Narratives](https://mlsanity.com/paper/2501.06478/) (2025)
- [SSVD: Structured SVD for Parameter-Efficient Fine-Tuning and Benchmarking under Domain Shift in ASR](https://mlsanity.com/paper/2509.02830/) (2025)
- [Can Generic LLMs Help Analyze Child-adult Interactions Involving Children with Autism in Clinical Observation?](https://mlsanity.com/paper/2411.10761/) (2024)
- [Improving Child Speech Recognition and Reading Mistake Detection by Using Prompts](https://mlsanity.com/paper/2506.11079/) (2025)
- [TICL: Text-Embedding KNN For Speech In-Context Learning Unlocks Speech Recognition Abilities of Large Multimodal Models](https://mlsanity.com/paper/2509.13395/) (2025)
- [Finding My Voice: Generative Reconstruction of Disordered Speech for Automated Clinical Evaluation](https://mlsanity.com/paper/2509.19231/) (2025)
- [Mind the Shift: Using Delta SSL Embeddings to Enhance Child ASR](https://mlsanity.com/paper/2601.20142/) (2026)
- [Large Language Models based ASR Error Correction for Child Conversations](https://mlsanity.com/paper/2505.16212/) (2025)
- [The Sound of Syntax: Finetuning and Comprehensive Evaluation of Language Models for Speech Pathology](https://mlsanity.com/paper/2509.16765/) (2025)
- [Egocentric Speaker Classification in Child-Adult Dyadic Interactions: From Sensing to Computational Modeling](https://mlsanity.com/paper/2409.09340/) (2024)
- [Evaluation of Speech Foundation Models for ASR on Child-Adult Conversations in Autism Diagnostic Sessions](https://mlsanity.com/paper/2409.16135/) (2024)
- [Detecting Children with Autism Spectrum Disorder based on Script-Centric Behavior Understanding with Emotional Enhancement](https://mlsanity.com/paper/2411.09413/) (2024)
- [SimClass: A Classroom Speech Dataset Generated via Game Engine Simulation For Automatic Speech Recognition Research](https://mlsanity.com/paper/2506.09206/) (2025)
- [K-Function: Joint Pronunciation Transcription and Feedback for Evaluating Kids Language Function](https://mlsanity.com/paper/2507.03043/) (2025)
- [Entropy-Aware Domain-Routed Mixture-of-Experts Speech-LLM Framework: A Case Study of Multi-Domain Child-Adult ASR](https://mlsanity.com/paper/2606.10454/) (2026)
- [Gumbel-BEARD: Automatic Layer Selection for Self-Supervised Adaptation of Whisper in Low-Resource Domains](https://mlsanity.com/paper/2606.11429/) (2026)
- [Examining Test-Time Adaptation for Personalized Child Speech Recognition](https://mlsanity.com/paper/2409.13095/) (2024)
- [From Weak Labels to Strong Results: Utilizing 5,000 Hours of Noisy Classroom Transcripts with Minimal Accurate Data](https://mlsanity.com/paper/2505.17088/) (2025)
- [Causal Structure Discovery for Error Diagnostics of Children's ASR](https://mlsanity.com/paper/2506.00402/) (2025)
- [Audio-3DVG: Unified Audio -- Point Cloud Fusion for 3D Visual Grounding](https://mlsanity.com/paper/2507.00669/) (2025)
- [RealClass: A Framework for Classroom Speech Simulation with Public Datasets and Game Engines](https://mlsanity.com/paper/2510.01462/) (2025)
- [Abjad-Kids: An Arabic Speech Classification Dataset for Primary Education](https://mlsanity.com/paper/2603.20255/) (2026)
- [Child-Centric Voice Anonymization in Single and Multi-Speaker Speech via Domain-Adapted SSL Models](https://mlsanity.com/paper/2606.29897/) (2026)
- [Encoder Awakening via Adapters: Effective Domain-Adaptive Fine-tuning of Speech-LLMs](https://mlsanity.com/paper/2609.17981/) (2026)
- [AURA: Uncertainty-Routed Activation Editing for Acoustic Grounding in Speech Foundation Models](https://mlsanity.com/paper/2609.23979/) (2026)
- [Deep Learning Techniques for Phoneme Recognition in Italian Children' s Speech](https://mlsanity.com/paper/2609.33060/) (2026)
- [Child-Adapted Structured Phonological Representations for Interpretable Speech Sound Analysis](https://mlsanity.com/paper/2610.00852/) (2026)

## Abstract (arXiv)

Speech foundation models (SFMs) have achieved state-of-the-art results for various speech tasks in supervised (e.g. Whisper) or self-supervised systems (e.g. WavLM). However, the performance of SFMs for child ASR has not been systematically studied. In addition, there is no benchmark for child ASR with standard evaluations, making the comparisons of novel ideas difficult. In this paper, we initiate and present a comprehensive benchmark on several child speech databases based on various SFMs (Whisper, Wav2vec2.0, HuBERT, and WavLM). Moreover, we investigate finetuning strategies by comparing various data augmentation and parameter-efficient finetuning (PEFT) methods. We observe that the behaviors of these methods are different when the model size increases. For example, PEFT matches the performance of full finetuning for large models but worse for small models. To stabilize finetuning using augmented data, we propose a perturbation invariant finetuning (PIF) loss as a regularization.