# RNN-T For Latency Controlled ASR With Improved Beam Search

arXiv: https://arxiv.org/abs/1911.01629 · HTML: https://mlsanity.com/paper/1911.01629/

## Key facts

- Cited by 20 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 5 Nov 2019.

## Built on by (most cited first, 20 of 20)

- [End-to-End Speech Recognition: A Survey](https://mlsanity.com/paper/2303.03329/) (2023)
- [Canary-1B-v2 & Parakeet-TDT-0.6B-v3: Efficient and High-Performance Models for Multilingual ASR and AST](https://mlsanity.com/paper/2509.14128/) (2025)
- [Open-Source Conversational AI with SpeechBrain 1.0](https://mlsanity.com/paper/2407.00463/) (2024)
- [Advancing RNN Transducer Technology for Speech Recognition](https://mlsanity.com/paper/2103.09935/) (2021)
- [Internal Language Model Training for Domain-Adaptive End-to-End Speech Recognition](https://mlsanity.com/paper/2102.01380/) (2021)
- [Factorized Neural Transducer for Efficient Language Model Adaptation](https://mlsanity.com/paper/2110.01500/) (2021)
- [Dissecting User-Perceived Latency of On-Device E2E Speech Recognition](https://mlsanity.com/paper/2104.02207/) (2021)
- [Modular Hybrid Autoregressive Transducer](https://mlsanity.com/paper/2210.17049/) (2022)
- [Knowledge Distillation for Neural Transducers from Large Self-Supervised Pre-trained Models](https://mlsanity.com/paper/2110.03334/) (2021)
- [Fast and parallel decoding for transducer](https://mlsanity.com/paper/2211.00484/) (2022)
- [Streaming End-to-End ASR based on Blockwise Non-Autoregressive Models](https://mlsanity.com/paper/2107.09428/) (2021)
- [JEIT: Joint End-to-End Model and Internal Language Model Training for Speech Recognition](https://mlsanity.com/paper/2302.08583/) (2023)
- [Semi-Autoregressive Streaming ASR With Label Context](https://mlsanity.com/paper/2309.10926/) (2023)
- [Pushing the Limits of Beam Search Decoding for Transducer-based ASR models](https://mlsanity.com/paper/2506.00185/) (2025)
- [Blank Collapse: Compressing CTC emission for the faster decoding](https://mlsanity.com/paper/2210.17017/) (2022)
- [Text Injection for Neural Contextual Biasing](https://mlsanity.com/paper/2406.02921/) (2024)
- [HAINAN: Fast and Accurate Transducer for Hybrid-Autoregressive ASR](https://mlsanity.com/paper/2410.02597/) (2024)
- [Combining Language Models For Specialized Domains: A Colorful Approach](https://mlsanity.com/paper/2310.19708/) (2023)
- [Weak Alignment Supervision from Hybrid Model Improves End-to-end ASR](https://mlsanity.com/paper/2311.14835/) (2023)
- [Alternating Weak Triphone/BPE Alignment Supervision from Hybrid Model Improves End-to-End ASR](https://mlsanity.com/paper/2402.15594/) (2024)

## Abstract (arXiv)

Neural transducer-based systems such as RNN Transducers (RNN-T) for automatic speech recognition (ASR) blend the individual components of a traditional hybrid ASR systems (acoustic model, language model, punctuation model, inverse text normalization) into one single model. This greatly simplifies training and inference and hence makes RNN-T a desirable choice for ASR systems. In this work, we investigate use of RNN-T in applications that require a tune-able latency budget during inference time. We also improved the decoding speed of the originally proposed RNN-T beam search algorithm. We evaluated our proposed system on English videos ASR dataset and show that neural RNN-T models can achieve comparable WER and better computational efficiency compared to a well tuned hybrid ASR baseline.