# Less Is More: Improved RNN-T Decoding Using Limited Label Context and Path Merging

arXiv: https://arxiv.org/abs/2012.06749 · HTML: https://mlsanity.com/paper/2012.06749/

## Key facts

- Cited by 26 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 12 Dec 2020.

## Built on by (most cited first, 26 of 26)

- [Recent Advances in End-to-End Automatic Speech Recognition](https://mlsanity.com/paper/2111.01690/) (2021)
- [End-to-End Speech Recognition: A Survey](https://mlsanity.com/paper/2303.03329/) (2023)
- [N-best T5: Robust ASR Error Correction using Multiple Input Hypotheses and Constrained Decoding Space](https://mlsanity.com/paper/2303.00456/) (2023)
- [Tied & Reduced RNN-T Decoder](https://mlsanity.com/paper/2109.07513/) (2021)
- [E2E Segmenter: Joint Segmenting and Decoding for Long-Form ASR](https://mlsanity.com/paper/2204.10749/) (2022)
- [Efficient Training of Neural Transducer for Speech Recognition](https://mlsanity.com/paper/2204.10586/) (2022)
- [Large-scale Language Model Rescoring on Long-form Data](https://mlsanity.com/paper/2306.08133/) (2023)
- [Improving Rare Word Recognition with LM-aware MWER Training](https://mlsanity.com/paper/2204.07553/) (2022)
- [Fast and parallel decoding for transducer](https://mlsanity.com/paper/2211.00484/) (2022)
- [Input Length Matters: Improving RNN-T and MWER Training for Long-form Telephony Speech Recognition](https://mlsanity.com/paper/2110.03841/) (2021)
- [A Language Agnostic Multilingual Streaming On-Device ASR System](https://mlsanity.com/paper/2208.13916/) (2022)
- [Extreme Encoder Output Frame Rate Reduction: Improving Computational Latencies of Large End-to-End Models](https://mlsanity.com/paper/2402.17184/) (2024)
- [Two-pass Decoding and Cross-adaptation Based System Combination of End-to-end Conformer and Hybrid TDNN ASR Systems](https://mlsanity.com/paper/2206.11596/) (2022)
- [VQ-T: RNN Transducers using Vector-Quantized Prediction Network States](https://mlsanity.com/paper/2208.01818/) (2022)
- [E2E Segmentation in a Two-Pass Cascaded Encoder ASR Model](https://mlsanity.com/paper/2211.15432/) (2022)
- [Lattice-Free Sequence Discriminative Training for Phoneme-Based Neural Transducers](https://mlsanity.com/paper/2212.04325/) (2022)
- [UML: A Universal Monolingual Output Layer for Multilingual ASR](https://mlsanity.com/paper/2302.11186/) (2023)
- [A Token-Wise Beam Search Algorithm for RNN-T](https://mlsanity.com/paper/2302.14357/) (2023)
- [Contextual Density Ratio for Language Model Biasing of Sequence to Sequence ASR Systems](https://mlsanity.com/paper/2206.14623/) (2022)
- [Semantic Segmentation with Bidirectional Language Models Improves Long-form ASR](https://mlsanity.com/paper/2305.18419/) (2023)
- [On the Prediction Network Architecture in RNN-T for ASR](https://mlsanity.com/paper/2206.14618/) (2022)
- [On the Relation between Internal Language Model and Sequence Discriminative Training for Neural Transducers](https://mlsanity.com/paper/2309.14130/) (2023)
- [Streaming Speech-to-Confusion Network Speech Recognition](https://mlsanity.com/paper/2306.03778/) (2023)
- [Globally Normalising the Transducer for Streaming Speech Recognition](https://mlsanity.com/paper/2307.10975/) (2023)
- [Investigating the Effect of Language Models in Sequence Discriminative Training for Neural Transducers](https://mlsanity.com/paper/2310.07345/) (2023)
- [ASR-Synchronized Speaker-Role Diarization](https://mlsanity.com/paper/2507.17765/) (2025)

## Abstract (arXiv)

End-to-end models that condition the output label sequence on all previously predicted labels have emerged as popular alternatives to conventional systems for automatic speech recognition (ASR). Since unique label histories correspond to distinct models states, such models are decoded using an approximate beam-search process which produces a tree of hypotheses. In this work, we study the influence of the amount of label context on the model's accuracy, and its impact on the efficiency of the decoding process. We find that we can limit the context of the recurrent neural network transducer (RNN-T) during training to just four previous word-piece labels, without degrading word error rate (WER) relative to the full-context baseline. Limiting context also provides opportunities to improve the efficiency of the beam-search process during decoding by removing redundant paths from the active beam, and instead retaining them in the final lattice. This path-merging scheme can also be applied when decoding the baseline full-context model through an approximation. Overall, we find that the proposed path-merging scheme is extremely effective allowing us to improve oracle WERs by up to 36% over the baseline, while simultaneously reducing the number of model evaluations by up to 5.3% without any degradation in WER.