# Personalized Speech recognition on mobile devices

arXiv: https://arxiv.org/abs/1603.03185 · HTML: https://mlsanity.com/paper/1603.03185/

## Key facts

- Cited by 23 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 10 Mar 2016.

## Built on by (most cited first, 23 of 23)

- [Federated Evaluation and Tuning for On-Device Personalization: System Design & Applications](https://mlsanity.com/paper/2102.08503/) (2021)
- [Personalization Strategies for End-to-End Speech Recognition Systems](https://mlsanity.com/paper/2102.07739/) (2021)
- [Fast Contextual Adaptation with Neural Associative Memory for On-Device Personalized Speech Recognition](https://mlsanity.com/paper/2110.02220/) (2021)
- [Dynamic Encoder Transducer: A Flexible Solution For Trading Off Accuracy For Latency](https://mlsanity.com/paper/2104.02176/) (2021)
- [Teaching keyword spotters to spot new keywords with limited examples](https://mlsanity.com/paper/2106.02443/) (2021)
- [Backdoor Attacks in Peer-to-Peer Federated Learning](https://mlsanity.com/paper/2301.09732/) (2023)
- [Memory-efficient Speech Recognition on Smart Devices](https://mlsanity.com/paper/2102.11531/) (2021)
- [On the quantization of recurrent neural networks](https://mlsanity.com/paper/2101.05453/) (2021)
- [Deferred NAM: Low-latency Top-K Context Injection via Deferred Context Encoding for Non-Streaming ASR](https://mlsanity.com/paper/2404.10180/) (2024)
- [Speech Understanding on Tiny Devices with A Learning Cache](https://mlsanity.com/paper/2311.18188/) (2023)
- [Text Injection for Neural Contextual Biasing](https://mlsanity.com/paper/2406.02921/) (2024)
- [Full-Rank No More: Low-Rank Weight Training for Modern Speech Recognition Models](https://mlsanity.com/paper/2410.07771/) (2024)
- [Deep CLAS: Deep Contextual Listen, Attend and Spell](https://mlsanity.com/paper/2409.17603/) (2024)
- [Towards Continual Entity Learning in Language Models for Conversational Agents](https://mlsanity.com/paper/2108.00082/) (2021)
- [Implementation Of Tiny Machine Learning Models On Arduino 33 BLE For Gesture And Speech Recognition](https://mlsanity.com/paper/2207.12866/) (2022)
- [An Experimental Study on Private Aggregation of Teacher Ensemble Learning for End-to-End Speech Recognition](https://mlsanity.com/paper/2210.05614/) (2022)
- [Enhancing Quantum Security over Federated Learning via Post-Quantum Cryptography](https://mlsanity.com/paper/2409.04637/) (2024)
- [PMF-CEC: Phoneme-augmented Multimodal Fusion for Context-aware ASR Error Correction with Error-specific Selective Decoding](https://mlsanity.com/paper/2506.11064/) (2025)
- [Adversarially Robust and Explainable Model Compression with On-Device Personalization for Text Classification](https://mlsanity.com/paper/2101.05624/) (2021)
- [Handling Compounding in Mobile Keyboard Input](https://mlsanity.com/paper/2201.06469/) (2022)
- [Deep learning model compression using network sensitivity and gradients](https://mlsanity.com/paper/2210.05111/) (2022)
- [Towards Personalization of CTC Speech Recognition Models with Contextual Adapters and Adaptive Boosting](https://mlsanity.com/paper/2210.09510/) (2022)
- [Not All Speech Is Intent: Adaptive Self-Correcting Inference Layer for Post-ASR False Wake-Up](https://mlsanity.com/paper/2609.12469/) (2026)

## Abstract (arXiv)

We describe a large vocabulary speech recognition system that is accurate, has low latency, and yet has a small enough memory and computational footprint to run faster than real-time on a Nexus 5 Android smartphone. We employ a quantized Long Short-Term Memory (LSTM) acoustic model trained with connectionist temporal classification (CTC) to directly predict phoneme targets, and further reduce its memory footprint using an SVD-based compression scheme. Additionally, we minimize our memory footprint by using a single language model for both dictation and voice command domains, constructed using Bayesian interpolation. Finally, in order to properly handle device-specific information, such as proper names and other context-dependent information, we inject vocabulary items into the decoder graph and bias the language model on-the-fly. Our system achieves 13.5% word error rate on an open-ended dictation task, running with a median speed that is seven times faster than real-time.