mlsanityData to 7 Oct 2026

Improving RNN-Transducers with Acoustic LookAhead

arXiv 2307.05006 ↗First posted 11 Jul 2023cs.CL cs.LG eess.AScited by 4

Not on the map. Why?

Abstract (arXiv)

RNN-Transducers (RNN-Ts) have gained widespread acceptance as an end-to-end model for speech to text conversion because of their high accuracy and streaming capabilities. A typical RNN-T independently encodes the input audio and the text context, and combines the two encodings by a thin joint network. While this architecture provides SOTA streaming accuracy, it also makes the model vulnerable to strong LM biasing which manifests as multi-step hallucination of text without acoustic evidence. In this paper we propose LookAhead that makes text representations more acoustically grounded by looking ahead into the future within the audio input. This technique yields a significant 5%-20% relative reduction in word error rate on both in-domain and out-of-domain evaluation sets.

Key facts. Cited by 4 papers in the map (data through 7 Oct 2026). Builds on 26 of its 39 references that are in the map. First posted on arXiv 11 Jul 2023.

Builds on

26 of 39 references.

References · 26

  1. 2017Attention Is All You Needcited by 59,932
  2. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  3. 2022Survey of Hallucination in Natural Language Generationcited by 2,056
  4. 2020Conformer: Convolution-augmented Transformer for Speech Recognitioncited by 1,838
  5. 2019Common Voice: A Massively-Multilingual Speech Corpuscited by 1,358
  6. 2018ESPnet: End-to-End Speech Processing Toolkitcited by 554
  7. 2020Transformer Transducer: A Streamable Speech Recognition Model with Transformer Encoders and RNN-T Losscited by 232
  8. 2018Streaming End-to-end Speech Recognition For Mobile Devicescited by 230
  9. 2015On Using Monolingual Corpora in Neural Machine Translationcited by 149
  10. 2020Emformer: Efficient Memory Transformer Based Acoustic Model For Low Latency Streaming Speech Recognitioncited by 111
  11. 2020Hybrid Autoregressive Transducer (hat)cited by 106
  12. 2020ContextNet: Improving Convolutional Neural Networks for Automatic Speech Recognition with Global Contextcited by 96
14 more
  1. 2018Exploring Architectures, Data and Units For Streaming End-to-End Speech Recognition with RNN-Transducercited by 95
  2. 2020Developing Real-time Streaming Transformer Transducer for Speech Recognition on Large-scale Datasetcited by 94
  3. 2020Internal Language Model Estimation for Domain-Adaptive End-to-End Speech Recognitioncited by 80
  4. 2020A Density Ratio Approach to Language Model Fusion in End-To-End Automatic Speech Recognitioncited by 73
  5. 2020Deep Shallow Fusion for RNN-T Personalizationcited by 64
  6. 2021Advancing RNN Transducer Technology for Speech Recognitioncited by 44
  7. 2021Factorized Neural Transducer for Efficient Language Model Adaptationcited by 31
  8. 2020RNN-T Models Fail to Generalize to Out-of-Domain Audio: Causes and Solutionscited by 26
  9. 2021Fast Text-Only Domain Adaptation of RNN-Transducer Prediction Networkcited by 19
  10. 2022Improving the fusion of acoustic and text representations in RNN-Tcited by 11
  11. 2020Improving accuracy of rare words for RNN-Transducer through unigram shallow fusioncited by 6
  12. 2023Lookahead When It Matters: Adaptive Non-causal Transformers for Streaming Neural Transducerscited by 6
  13. 2022Adaptive Discounting of Implicit Language Models in RNN-Transducerscited by 5
  14. 2022VQ-T: RNN Transducers using Vector-Quantized Prediction Network Statescited by 3

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 1; 2025: 1; 2026: 2.

4 papers.

  1. 2024LINOCS: Lookahead Inference of Networked Operators for Continuous Stabilitycited by 1
  2. 2026Lost in Transcription: How Speech-to-Text Errors Derail Code Understandingcited by 1
  3. 2025CodeVaani: A Multilingual, Voice-Based Code Learning Assistantcited by 0
  4. 2026The Anatomy of an ASR Hallucinationcited by 0