# Improving RNN-T ASR Accuracy Using Context Audio

arXiv: https://arxiv.org/abs/2011.10538 · HTML: https://mlsanity.com/paper/2011.10538/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 20 Nov 2020.

## Built on by (most cited first, 8 of 8)

- [Recent Advances in End-to-End Automatic Speech Recognition](https://mlsanity.com/paper/2111.01690/) (2021)
- [Accelerating Transducers through Adjacent Token Merging](https://mlsanity.com/paper/2306.16009/) (2023)
- [Updated Corpora and Benchmarks for Long-Form Speech Recognition](https://mlsanity.com/paper/2309.15013/) (2023)
- [LongFNT: Long-form Speech Recognition with Factorized Neural Transducer](https://mlsanity.com/paper/2211.09412/) (2022)
- [Personalized Predictive ASR for Latency Reduction in Voice Assistants](https://mlsanity.com/paper/2305.13794/) (2023)
- [Using Large Language Model for End-to-End Chinese ASR and NER](https://mlsanity.com/paper/2401.11382/) (2024)
- [Contextual-Utterance Training for Automatic Speech Recognition](https://mlsanity.com/paper/2210.16238/) (2022)
- [Beyond Fresh Starts: Stateful Inference for Streaming ASR in Conversational Voice Agents](https://mlsanity.com/paper/2608.22101/) (2026)

## Abstract (arXiv)

We present a training scheme for streaming automatic speech recognition (ASR) based on recurrent neural network transducers (RNN-T) which allows the encoder network to learn to exploit context audio from a stream, using segmented or partially labeled sequences of the stream during training. We show that the use of context audio during training and inference can lead to word error rate reductions of more than 6% in a realistic production setting for a voice assistant ASR system. We investigate the effect of the proposed training approach on acoustically challenging data containing background speech and present data points which indicate that this approach helps the network learn both speaker and environment adaptation. To gain further insight into the ability of a long short-term memory (LSTM) based ASR encoder to exploit long-term context, we also visualize RNN-T loss gradients with respect to the input.