# Adaptation of Whisper models to child speech recognition

arXiv: https://arxiv.org/abs/2307.13008 · HTML: https://mlsanity.com/paper/2307.13008/

## Key facts

- Cited by 26 papers in the map (data through 14 Oct 2026).
- Builds on 15 of its 33 references that are in the map.
- First posted on arXiv 24 Jul 2023.

## Builds on (most cited first, 15 of 15)

- [Robust Speech Recognition via Large-Scale Weak Supervision](https://mlsanity.com/paper/2212.04356/) (2022)
- [wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representations](https://mlsanity.com/paper/2006.11477/) (2020)
- [Conformer: Convolution-augmented Transformer for Speech Recognition](https://mlsanity.com/paper/2005.08100/) (2020)
- [LibriTTS: A Corpus Derived from LibriSpeech for Text-to-Speech](https://mlsanity.com/paper/1904.02882/) (2019)
- [Libri-Light: A Benchmark for ASR with Limited or No Supervision](https://mlsanity.com/paper/1912.07875/) (2019)
- [QuartzNet: Deep Automatic Speech Recognition with 1D Time-Channel Separable Convolutions](https://mlsanity.com/paper/1910.10261/) (2019)
- [SpeechStew: Simply Mix All Available Speech Recognition Data to Train One Large Neural Network](https://mlsanity.com/paper/2104.02133/) (2021)
- [Toward domain-invariant speech recognition via large scale training](https://mlsanity.com/paper/1808.05312/) (2018)
- [Towards Better Domain Adaptation for Self-supervised Models: A Case Study of Child ASR](https://mlsanity.com/paper/2305.00115/) (2023)
- [Transfer Learning from Adult to Children for Speech Recognition: Evaluation, Analysis and Recommendations](https://mlsanity.com/paper/1805.03322/) (2018)
- [DRAFT: A Novel Framework to Reduce Domain Shifting in Self-supervised Learning and Its Application to Children's ASR](https://mlsanity.com/paper/2206.07931/) (2022)
- [End-to-End Neural Systems for Automatic Children Speech Recognition: An Empirical Study](https://mlsanity.com/paper/2102.09918/) (2021)
- [A Wav2vec2-Based Experimental Study on Self-Supervised Learning Methods to Improve Child Speech Recognition](https://mlsanity.com/paper/2204.05419/) (2022)
- [Fundamental Frequency Feature Normalization and Data Augmentation for Child Speech Recognition](https://mlsanity.com/paper/2102.09106/) (2021)
- [Transfer Learning for Robust Low-Resource Children's Speech ASR with Transformers and Source-Filter Warping](https://mlsanity.com/paper/2206.09396/) (2022)

## Built on by (most cited first, 26 of 26)

- [Benchmarking Children's ASR with Supervised and Self-supervised Speech Foundation Models](https://mlsanity.com/paper/2406.10507/) (2024)
- [Kid-Whisper: Towards Bridging the Performance Gap in Automatic Speech Recognition for Children VS. Adults](https://mlsanity.com/paper/2309.07927/) (2023)
- [CPT-Boosted Wav2vec2.0: Towards Noise Robust Speech Recognition for Classroom Environments](https://mlsanity.com/paper/2409.14494/) (2024)
- [Adapting Whisper for Parameter-efficient Code-Switching Speech Recognition via Soft Prompt Tuning](https://mlsanity.com/paper/2506.21576/) (2025)
- [Continued Pretraining for Domain Adaptation of Wav2vec2.0 in Automatic Speech Recognition for Elementary Math Classroom Settings](https://mlsanity.com/paper/2405.13018/) (2024)
- [TICL+: A Case Study On Speech In-Context Learning for Children's Speech Recognition](https://mlsanity.com/paper/2512.18263/) (2025)
- [Using fine-tuning and min lookahead beam search to improve Whisper](https://mlsanity.com/paper/2309.10299/) (2023)
- [Automatic Speech Recognition (ASR) for the Diagnosis of pronunciation of Speech Sound Disorders in Korean children](https://mlsanity.com/paper/2403.08187/) (2024)
- [FASA: a Flexible and Automatic Speech Aligner for Extracting High-quality Aligned Children Speech Data](https://mlsanity.com/paper/2406.17926/) (2024)
- [Arabic Little STT: Arabic Children Speech Recognition Dataset](https://mlsanity.com/paper/2510.23319/) (2025)
- [A comparative analysis between Conformer-Transducer, Whisper, and wav2vec2 for improving the child speech recognition](https://mlsanity.com/paper/2311.04936/) (2023)
- [Efficient Compression of Multitask Multilingual Speech Models](https://mlsanity.com/paper/2405.00966/) (2024)
- [Language-Aware Prompt Tuning for Parameter-Efficient Seamless Language Expansion in Multilingual ASR](https://mlsanity.com/paper/2506.21577/) (2025)
- [Evaluation of Speech Foundation Models for ASR on Child-Adult Conversations in Autism Diagnostic Sessions](https://mlsanity.com/paper/2409.16135/) (2024)
- [Automatic Speech Recognition for the Ika Language](https://mlsanity.com/paper/2410.00940/) (2024)
- [Automated evaluation of children's speech fluency for low-resource languages](https://mlsanity.com/paper/2505.19671/) (2025)
- [Using State-of-the-Art Speech Models to Evaluate Oral Reading Fluency in Ghana](https://mlsanity.com/paper/2310.17606/) (2023)
- [Examining Test-Time Adaptation for Personalized Child Speech Recognition](https://mlsanity.com/paper/2409.13095/) (2024)
- [Benchmarking NLP-supported Language Sample Analysis for Swiss Children's Speech](https://mlsanity.com/paper/2504.00780/) (2025)
- [Causal Structure Discovery for Error Diagnostics of Children's ASR](https://mlsanity.com/paper/2506.00402/) (2025)
- [Which one Performs Better? Wav2Vec or Whisper? Applying both in Badini Kurdish Speech to Text (BKSTT)](https://mlsanity.com/paper/2508.09957/) (2025)
- [InterventionLens: A Multi-Agent Framework for Detecting ASD Intervention Strategies in Parent-Child Shared Reading](https://mlsanity.com/paper/2603.13710/) (2026)
- [Utterance-Level Methods for Identifying Reliable ASR-Output for Child Speech](https://mlsanity.com/paper/2604.19801/) (2026)
- [From Review to Design: Ethical Multimodal Driver Monitoring Systems for Risk Mitigation, Incident Response, and Accountability in Automated Vehicles](https://mlsanity.com/paper/2605.06439/) (2026)
- [Transcribing Children's Speech: ASR Performance and Obtaining Reliable Orthographic Transcriptions](https://mlsanity.com/paper/2605.28833/) (2026)
- [Child ASR Adaptation with Adult Retention: An Empirical Study](https://mlsanity.com/paper/2610.08827/) (2026)

## Abstract (arXiv)

Automatic Speech Recognition (ASR) systems often struggle with transcribing child speech due to the lack of large child speech datasets required to accurately train child-friendly ASR models. However, there are huge amounts of annotated adult speech datasets which were used to create multilingual ASR models, such as Whisper. Our work aims to explore whether such models can be adapted to child speech to improve ASR for children. In addition, we compare Whisper child-adaptations with finetuned self-supervised models, such as wav2vec2. We demonstrate that finetuning Whisper on child speech yields significant improvements in ASR performance on child speech, compared to non finetuned Whisper models. Additionally, utilizing self-supervised Wav2vec2 models that have been finetuned on child speech outperforms Whisper finetuning.