# Memory-efficient Speech Recognition on Smart Devices

arXiv: https://arxiv.org/abs/2102.11531 · HTML: https://mlsanity.com/paper/2102.11531/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- Builds on 9 of its 25 references that are in the map.
- First posted on arXiv 23 Feb 2021.

## Builds on (most cited first, 9 of 9)

- [Layer Normalization](https://mlsanity.com/paper/1607.06450/) (2016)
- [Reducing Transformer Depth on Demand with Structured Dropout](https://mlsanity.com/paper/1909.11556/) (2019)
- [Efficient Processing of Deep Neural Networks: A Tutorial and Survey](https://mlsanity.com/paper/1703.09039/) (2017)
- [LSTM: A Search Space Odyssey](https://mlsanity.com/paper/1503.04069/) (2015)
- [Streaming End-to-end Speech Recognition For Mobile Devices](https://mlsanity.com/paper/1811.06621/) (2018)
- [Improving RNN Transducer Modeling for End-to-End Speech Recognition](https://mlsanity.com/paper/1909.12415/) (2019)
- [Optimizing Speech Recognition For The Edge](https://mlsanity.com/paper/1909.12408/) (2019)
- [Personalized Speech recognition on mobile devices](https://mlsanity.com/paper/1603.03185/) (2016)
- [Residual LSTM: Design of a Deep Recurrent Architecture for Distant Speech Recognition](https://mlsanity.com/paper/1701.03360/) (2017)

## Built on by (most cited first, 8 of 8)

- [PARP: Prune, Adjust and Re-Prune for Self-Supervised Speech Recognition](https://mlsanity.com/paper/2106.05933/) (2021)
- [From English to More Languages: Parameter-Efficient Model Reprogramming for Cross-Lingual Speech Recognition](https://mlsanity.com/paper/2301.07851/) (2023)
- [PhiNets: a scalable backbone for low-power AI at the edge](https://mlsanity.com/paper/2110.00337/) (2021)
- [Multi-stage Progressive Compression of Conformer Transducer for On-device Speech Recognition](https://mlsanity.com/paper/2210.00169/) (2022)
- [Losses Can Be Blessings: Routing Self-Supervised Speech Representations Towards Efficient Multilingual and Multitask Speech Processing](https://mlsanity.com/paper/2211.01522/) (2022)
- [Breaking Down Power Barriers in On-Device Streaming ASR: Insights and Solutions](https://mlsanity.com/paper/2402.13076/) (2024)
- [Two-Pass End-to-End ASR Model Compression](https://mlsanity.com/paper/2201.02741/) (2022)
- [Flexi-Transducer: Optimizing Latency, Accuracy and Compute forMulti-Domain On-Device Scenarios](https://mlsanity.com/paper/2104.02232/) (2021)

## Abstract (arXiv)

Recurrent transducer models have emerged as a promising solution for speech recognition on the current and next generation smart devices. The transducer models provide competitive accuracy within a reasonable memory footprint alleviating the memory capacity constraints in these devices. However, these models access parameters from off-chip memory for every input time step which adversely effects device battery life and limits their usability on low-power devices. We address transducer model's memory access concerns by optimizing their model architecture and designing novel recurrent cell designs. We demonstrate that i) model's energy cost is dominated by accessing model weights from off-chip memory, ii) transducer model architecture is pivotal in determining the number of accesses to off-chip memory and just model size is not a good proxy, iii) our transducer model optimizations and novel recurrent cell reduces off-chip memory accesses by 4.5x and model size by 2x with minimal accuracy impact.