# RTMobile: Beyond Real-Time Mobile Acceleration of RNNs for Speech Recognition

arXiv: https://arxiv.org/abs/2002.11474 · HTML: https://mlsanity.com/paper/2002.11474/

## Key facts

- Cited by 19 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 19 Feb 2020.

## Built on by (most cited first, 19 of 19)

- [MEST: Accurate and Fast Memory-Economic Sparse Training Framework on the Edge](https://mlsanity.com/paper/2110.14032/) (2021)
- [DNNFusion: Accelerating Deep Neural Networks Execution with Advanced Operator Fusion](https://mlsanity.com/paper/2108.13342/) (2021)
- [SparCL: Sparse Continual Learning on the Edge](https://mlsanity.com/paper/2209.09476/) (2022)
- [DPHuBERT: Joint Distillation and Pruning of Self-Supervised Speech Models](https://mlsanity.com/paper/2305.17651/) (2023)
- [Achieving on-Mobile Real-Time Super-Resolution with Neural Architecture and Pruning Search](https://mlsanity.com/paper/2108.08910/) (2021)
- [Structured Pruning of Self-Supervised Pre-trained Models for Speech Recognition and Understanding](https://mlsanity.com/paper/2302.14132/) (2023)
- [Compiler-Aware Neural Architecture Search for On-Mobile Real-time Super-Resolution](https://mlsanity.com/paper/2207.12577/) (2022)
- [HALOC: Hardware-Aware Automatic Low-Rank Compression for Compact Neural Networks](https://mlsanity.com/paper/2301.09422/) (2023)
- [Quantum Neural Network Compression](https://mlsanity.com/paper/2207.01578/) (2022)
- [Layer Freezing & Data Sieving: Missing Pieces of a Generic Framework for Sparse Training](https://mlsanity.com/paper/2209.11204/) (2022)
- [GRIM: A General, Real-Time Deep Learning Inference Framework for Mobile Devices based on Fine-Grained Structured Weight Sparsity](https://mlsanity.com/paper/2108.11033/) (2021)
- [I3D: Transformer architectures with input-dependent dynamic depth for speech recognition](https://mlsanity.com/paper/2303.07624/) (2023)
- [Automatic Mapping of the Best-Suited DNN Pruning Schemes for Real-Time Mobile Acceleration](https://mlsanity.com/paper/2111.11581/) (2021)
- [All-in-One: A Highly Representative DNN Pruning Framework for Edge Devices with Dynamic Power Management](https://mlsanity.com/paper/2212.05122/) (2022)
- [A Comprehensive Review and a Taxonomy of Edge Machine Learning: Requirements, Paradigms, and Techniques](https://mlsanity.com/paper/2302.08571/) (2023)
- [Uncovering Critical Features for Deepfake Detection through the Lottery Ticket Hypothesis](https://mlsanity.com/paper/2507.15636/) (2025)
- [Achieving Real-Time Object Detection on MobileDevices with Neural Pruning Search](https://mlsanity.com/paper/2106.14943/) (2021)
- [Robust Group Anomaly Detection for Quasi-Periodic Network Time Series](https://mlsanity.com/paper/2506.16815/) (2025)
- [CoCoPIE XGen: A Full-Stack AI-Oriented Optimizing Framework](https://mlsanity.com/paper/2206.10620/) (2022)

## Abstract (arXiv)

Recurrent neural networks (RNNs) based automatic speech recognition has nowadays become prevalent on mobile devices such as smart phones. However, previous RNN compression techniques either suffer from hardware performance overhead due to irregularity or significant accuracy loss due to the preserved regularity for hardware friendliness. In this work, we propose RTMobile that leverages both a novel block-based pruning approach and compiler optimizations to accelerate RNN inference on mobile devices. Our proposed RTMobile is the first work that can achieve real-time RNN inference on mobile platforms. Experimental results demonstrate that RTMobile can significantly outperform existing RNN hardware acceleration methods in terms of inference accuracy and time. Compared with prior work on FPGA, RTMobile using Adreno 640 embedded GPU on GRU can improve the energy-efficiency by about 40× while maintaining the same inference time.