mlsanityData to 7 Oct 2026

MobileASR: A resource-aware on-device learning framework for user voice personalization applications on mobile phones

arXiv 2306.09384 ↗First posted 15 Jun 2023eess.AS cs.AIcited by 0

Not on the map. Why?

Abstract (arXiv)

We describe a comprehensive methodology for developing user-voice personalized automatic speech recognition (ASR) models by effectively training models on mobile phones, allowing user data and models to be stored and used locally. To achieve this, we propose a resource-aware sub-model-based training approach that considers the RAM, and battery capabilities of mobile phones. By considering the evaluation metric and resource constraints of the mobile phones, we are able to perform efficient training and halt the process accordingly. To simulate real users, we use speakers with various accents. The entire on-device training and evaluation framework was then tested on various mobile phones across brands. We show that fine-tuning the models and selecting the right hyperparameter values is a trade-off between the lowest achievable performance metric, on-device training time, and memory consumption. Overall, our methodology offers a comprehensive solution for developing personalized ASR models while leveraging the capabilities of mobile phones, and balancing the need for accuracy with resource constraints.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 19 of its 37 references that are in the map. First posted on arXiv 15 Jun 2023.

Builds on

19 of 37 references.

References · 19

  1. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  2. 2014Sequence to Sequence Learning with Neural Networkscited by 2,974
  3. 2020Conformer: Convolution-augmented Transformer for Speech Recognitioncited by 1,838
  4. 2019Common Voice: A Massively-Multilingual Speech Corpuscited by 1,358
  5. 2019Self-training with Noisy Student improves ImageNet classificationcited by 1,304
  6. 2015Listen, Attend and Spellcited by 542
  7. 2015Deep Speech 2: End-to-End Speech Recognition in English and Mandarincited by 449
  8. 2014Deep Speech: Scaling up end-to-end speech recognitioncited by 354
  9. 2020HeteroFL: Computation and Communication Efficient Federated Learning for Heterogeneous Clientscited by 334
  10. 2018Streaming End-to-end Speech Recognition For Mobile Devicescited by 230
  11. 2021FjORD: Fair and Accurate Federated Learning under heterogeneous targets with Ordered Dropoutcited by 192
  12. 2020Training BatchNorm and Only BatchNorm: On the Expressive Power of Random Features in CNNscited by 103
7 more
  1. 2022FedRolex: Model-Heterogeneous Federated Learning with Rolling Sub-Model Extractioncited by 102
  2. 2019Personalization of End-to-end Speech Recognition On Mobile Devices For Named Entitiescited by 41
  3. 2019Lessons from Building Acoustic Models with a Million Hours of Speechcited by 27
  4. 2019An Investigation Into On-device Personalization of End-to-end Automatic Speech Recognition Modelscited by 25
  5. 2020Low-rank Gradient Approximation For Memory-Efficient On-device Training of Deep Neural Networkcited by 20
  6. 2020Unsupervised Speaker Adaptation using Attention-based Speaker Memory for End-to-End ASRcited by 11
  7. 2021On-Device Personalization of Automatic Speech Recognition Models for Disordered Speechcited by 5

Built on by

Not cited yet.