Accelerator-Aware Training for Transducer-Based Speech Recognition
Not on the map. Why?
Abstract (arXiv)
Machine learning model weights and activations are represented in full-precision during training. This leads to performance degradation in runtime when deployed on neural network accelerator (NNA) chips, which leverage highly parallelized fixed-point arithmetic to improve runtime memory and latency. In this work, we replicate the NNA operators during the training phase, accounting for the degradation due to low-precision inference on the NNA in back-propagation. Our proposed method efficiently emulates NNA operations, thus foregoing the need to transfer quantization error-prone data to the Central Processing Unit (CPU), ultimately reducing the user perceived latency (UPL). We apply our approach to Recurrent Neural Network-Transducer (RNN-T), an attractive architecture for on-device streaming speech recognition tasks. We train and evaluate models on 270K hours of English data and show a 5-7% improvement in engine latency while saving up to 10% relative degradation in WER.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 20 of its 27 references that are in the map. First posted on arXiv 12 May 2023.
Builds on
20 of 27 references.
References · 20
- 2015Deep Compression: Compressing Deep Neural Networks with Pruning, Trained Quantization and Huffman Codingcited by 2,329
- 2013Estimating or Propagating Gradients Through Stochastic Neurons for Conditional Computationcited by 2,079
- 2017Quantization and Training of Neural Networks for Efficient Integer-Arithmetic-Only Inferencecited by 1,593
- 2017To prune, or not to prune: exploring the efficacy of pruning for model compressioncited by 603
- 2015LSTM: A Search Space Odysseycited by 316
- 2019Transformer-based Acoustic Modeling for Hybrid Speech Recognitioncited by 88
- 2020FastEmit: Low-latency Streaming ASR with Sequence-level Emission Regularizationcited by 69
- 2017Simple Recurrent Units for Highly Parallelizable Recurrencecited by 61
- 2020Efficient Knowledge Distillation for RNN-Transducer Modelscited by 38
- 2019Optimizing Speech Recognition For The Edgecited by 32
- 2020Exploring Transformers for Large-Scale Speech Recognitioncited by 24
- 2018Accelerator-Aware Pruning for Convolutional Neural Networkscited by 22
8 more
- 2016On the efficient representation and execution of deep acoustic modelscited by 13
- 2021CoDERT: Distilling Encoder Representations with Co-learning for Transducer-based Speech Recognitioncited by 11
- 2021Amortized Neural Networks for Low-Latency Speech Recognitioncited by 9
- 2021Sparsification via Compressed Sensing for Automatic Speech Recognitioncited by 8
- 2022ConvRNN-T: Convolutional Augmented Recurrent Neural Network Transducers for Streaming Speech Recognitioncited by 8
- 2021On the quantization of recurrent neural networkscited by 7
- 2021Collaborative Training of Acoustic Encoders for Speech Recognitioncited by 7
- 2021Bifocal Neural ASR: Exploiting Keyword Spotting for Inference Optimizationcited by 7
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2022Sub-8-Bit Quantization Aware Training for 8-Bit Neural Network Accelerator with On-Device Speech Recognition
- 2025A 71.2-μW Speech Recognition Accelerator with Recurrent Spiking Neural Network
- 2023TODM: Train Once Deploy Many Efficient Supernet-Based RNN-T Compression For On-device ASR Models
- 2022Vau da muntanialas: Energy-efficient multi-die scalable acceleration of RNN inference
- 2026Pushing the Limits of On-Device Streaming ASR: A Compact, High-Accuracy English Model for Low-Latency Inference
- 2022Accelerating Inference and Language Model Fusion of Recurrent Neural Network Transducers via End-to-End 4-bit Quantization
- 2025Accelerating Linear Recurrent Neural Networks for the Edge with Unstructured Sparsity
- 2022Accelerating RNN-based Speech Enhancement on a Multi-Core MCU with Mixed FP16-INT8 Post-Training Quantization