# On-Device Personalization of Automatic Speech Recognition Models for Disordered Speech

arXiv: https://arxiv.org/abs/2106.10259 · HTML: https://mlsanity.com/paper/2106.10259/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 10 of its 33 references that are in the map.
- First posted on arXiv 18 Jun 2021.

## Builds on (most cited first, 10 of 10)

- [Speech Recognition with Deep Recurrent Neural Networks](https://mlsanity.com/paper/1303.5778/) (2013)
- [Streaming End-to-end Speech Recognition For Mobile Devices](https://mlsanity.com/paper/1811.06621/) (2018)
- [On Using Monolingual Corpora in Neural Machine Translation](https://mlsanity.com/paper/1503.03535/) (2015)
- [Cold Fusion: Training Seq2Seq Models Together with Language Models](https://mlsanity.com/paper/1708.06426/) (2017)
- [A Comparison of Techniques for Language Model Integration in Encoder-Decoder Speech Recognition](https://mlsanity.com/paper/1807.10857/) (2018)
- [Personalizing ASR for Dysarthric and Accented Speech with Limited Data](https://mlsanity.com/paper/1907.13511/) (2019)
- [Personalization of End-to-end Speech Recognition On Mobile Devices For Named Entities](https://mlsanity.com/paper/1912.09251/) (2019)
- [Toward domain-invariant speech recognition via large scale training](https://mlsanity.com/paper/1808.05312/) (2018)
- [An Investigation Into On-device Personalization of End-to-end Automatic Speech Recognition Models](https://mlsanity.com/paper/1909.06678/) (2019)
- [Multi-Accent Adaptation based on Gate Mechanism](https://mlsanity.com/paper/2011.02774/) (2020)

## Built on by (most cited first, 5 of 5)

- [Personalized Automatic Speech Recognition Trained on Small Disordered Speech Datasets](https://mlsanity.com/paper/2110.04612/) (2021)
- [UserLibri: A Dataset for ASR Personalization Using Only Text](https://mlsanity.com/paper/2207.00706/) (2022)
- [Hypernetworks for Personalizing ASR to Atypical Speech](https://mlsanity.com/paper/2406.04240/) (2024)
- [Idiosyncratic Versus Normative Modeling of Atypical Speech Recognition: Dysarthric Case Studies](https://mlsanity.com/paper/2509.16718/) (2025)
- [MobileASR: A resource-aware on-device learning framework for user voice personalization applications on mobile phones](https://mlsanity.com/paper/2306.09384/) (2023)

## Abstract (arXiv)

While current state-of-the-art Automatic Speech Recognition (ASR) systems achieve high accuracy on typical speech, they suffer from significant performance degradation on disordered speech and other atypical speech patterns. Personalization of ASR models, a commonly applied solution to this problem, is usually performed in a server-based training environment posing problems around data privacy, delayed model-update times, and communication cost for copying data and models between mobile device and server infrastructure. In this paper, we present an approach to on-device based ASR personalization with very small amounts of speaker-specific data. We test our approach on a diverse set of 100 speakers with disordered speech and find median relative word error rate improvement of 71% with only 50 short utterances required per speaker. When tested on a voice-controlled home automation platform, on-device personalized models show a median task success rate of 81%, compared to only 40% of the unadapted models.