mlsanityData to 7 Oct 2026

Advancing RNN Transducer Technology for Speech Recognition

arXiv 2103.09935 ↗First posted 17 Mar 2021cs.CL cs.SD eess.AScited by 44

Not on the map. Why?

Abstract (arXiv)

We investigate a set of techniques for RNN Transducers (RNN-Ts) that were instrumental in lowering the word error rate on three different tasks (Switchboard 300 hours, conversational Spanish 780 hours and conversational Italian 900 hours). The techniques pertain to architectural changes, speaker adaptation, language model fusion, model combination and general training recipe. First, we introduce a novel multiplicative integration of the encoder and prediction network vectors in the joint network (as opposed to additive). Second, we discuss the applicability of i-vector speaker adaptation to RNN-Ts in conjunction with data perturbation. Third, we explore the effectiveness of the recently proposed density ratio language model fusion for these tasks. Last but not least, we describe the other components of our training recipe and their effect on recognition performance. We report a 5.9% and 12.5% word error rate on the Switchboard and CallHome test sets of the NIST Hub5 2000 evaluation and a 12.7% WER on the Mozilla CommonVoice Italian test set.

Key facts. Cited by 44 papers in the map (data through 7 Oct 2026). Builds on 21 of its 37 references that are in the map. First posted on arXiv 17 Mar 2021.

Builds on

21 of 37 references.

References · 21

  1. 2016Google's Neural Machine Translation System: Bridging the Gap between Human and Machine Translationcited by 1,471
  2. 2019SpecAugment: A Simple Data Augmentation Method for Automatic Speech Recognitioncited by 1,355
  3. 2013Speech Recognition with Deep Recurrent Neural Networkscited by 666
  4. 2015Listen, Attend and Spellcited by 542
  5. 2018Streaming End-to-end Speech Recognition For Mobile Devicescited by 230
  6. 2017State-of-the-art Speech Recognition With Sequence-to-Sequence Modelscited by 196
  7. 2015End-to-End Attention-based Large Vocabulary Speech Recognitioncited by 148
  8. 2020Hybrid Autoregressive Transducer (hat)cited by 106
  9. 2018Exploring Architectures, Data and Units For Streaming End-to-End Speech Recognition with RNN-Transducercited by 95
  10. 2020A Density Ratio Approach to Language Model Fusion in End-To-End Automatic Speech Recognitioncited by 73
  11. 2018SwitchOut: an Efficient Data Augmentation Algorithm for Neural Machine Translationcited by 72
  12. 2019Improving RNN Transducer Modeling for End-to-End Speech Recognitioncited by 68
9 more
  1. 2019Transformer-Transducer: End-to-End Speech Recognition with Self-Attentioncited by 68
  2. 2017Exploring Neural Transducers for End-to-End Speech Recognitioncited by 43
  3. 2020Single headed attention based sequence-to-sequence model for state-of-the-art results on Switchboardcited by 36
  4. 2020A New Training Pipeline for an Improved Neural Transducercited by 34
  5. 2020RNN-T Models Fail to Generalize to Out-of-Domain Audio: Causes and Solutionscited by 26
  6. 2019Minimum Bayes Risk Training of RNN-Transducer for End-to-End Speech Recognitioncited by 23
  7. 2019RNN-T For Latency Controlled ASR With Improved Beam Searchcited by 19
  8. 2016On Multiplicative Integration with Recurrent Neural Networkscited by 15
  9. 2018Exploring RNN-Transducer for Chinese Speech Recognitioncited by 8

Built on by

Cited by year

Citing papers per year: 2021: 12; 2022: 18; 2023: 6; 2024: 5; 2025: 3.

44 papers.

Sort:

By area:

  1. 2021Recent Advances in End-to-End Automatic Speech Recognitioncited by 138
  2. 2023End-to-End Speech Recognition: A Surveycited by 127
  3. 2021Scaling End-to-End Models for Large-Scale Multilingual ASRcited by 61
  4. 2022Large-Scale Streaming End-to-End Speech Translation with Neural Transducerscited by 31
  5. 2021On the limit of English conversational speech recognitioncited by 30
  6. 2021Residual Adapters for Parameter-Efficient ASR Adaptation to Atypical and Accented Speechcited by 30
  7. 2022Effect and Analysis of Large-scale Language Model Rescoring on Competitive ASR Systemscited by 23
  8. 2022Integrating Text Inputs For Training and Adapting RNN Transducer ASR Modelscited by 20
  9. 2022Towards Reducing the Need for Speech Training Data To Build Spoken Language Understanding Systemscited by 20
  10. 2021On Language Model Integration for RNN Transducer based Speech Recognitioncited by 17
  11. 2022Efficient Training of Neural Transducer for Speech Recognitioncited by 14
  12. 2023Accelerating Transducers through Adjacent Token Mergingcited by 14
32 more
  1. 2022A Weakly-Supervised Streaming Multilingual Speech Model with Truly Zero-Shot Capabilitycited by 12
  2. 2022Improving the fusion of acoustic and text representations in RNN-Tcited by 11
  3. 2021Input Length Matters: Improving RNN-T and MWER Training for Long-form Telephony Speech Recognitioncited by 10
  4. 2022Improving Deliberation by Text-Only and Semi-Supervised Trainingcited by 10
  5. 2021Integrating Dialog History into End-to-End Spoken Language Understanding Systemscited by 9
  6. 2022Improving the Training Recipe for a Robust Conformer-based Hybrid Modelcited by 9
  7. 2022Towards End-to-End Integration of Dialog History for Improved Spoken Language Understandingcited by 8
  8. 2024Recurrent Aligned Network for Generalized Pedestrian Trajectory Predictioncited by 8
  9. 2024Brain-inspired Artificial Intelligence: A Comprehensive Reviewcited by 8
  10. 2021RNN Transducer Models For Spoken Language Understandingcited by 7
  11. 20214-bit Quantization of LSTM-based Speech Recognition Modelscited by 7
  12. 2023CIF-T: A Novel CIF-based Transducer Architecture for Automatic Speech Recognitioncited by 6
  13. 2023Semi-Autoregressive Streaming ASR With Label Contextcited by 5
  14. 2021Advancing CTC-CRF Based End-to-End Speech Recognition with Wordpieces and Conformerscited by 4
  15. 2022A Deep Reinforcement Learning Blind AI in DareFightingICEcited by 4
  16. 2023Improving RNN-Transducers with Acoustic LookAheadcited by 4
  17. 2024Boosting keyword spotting through on-device learnable user speech characteristicscited by 4
  18. 2025HalluRNN: Mitigating Hallucinations via Recurrent Cross-Layer Reasoning in Large Vision-Language Modelscited by 4
  19. 2022Memory-Efficient Training of RNN-Transducer with Sampled Softmaxcited by 3
  20. 2022VQ-T: RNN Transducers using Vector-Quantized Prediction Network Statescited by 3
  21. 2021Reducing Exposure Bias in Training Recurrent Neural Network Transducerscited by 2
  22. 2021Towards efficient end-to-end speech recognition with biologically-inspired neural networkscited by 2
  23. 2022Improving Generalization of Deep Neural Network Acoustic Models with Length Perturbation and N-best Based Label Smoothingcited by 2
  24. 2022Accelerating Inference and Language Model Fusion of Recurrent Neural Network Transducers via End-to-End 4-bit Quantizationcited by 2
  25. 2023Multiple Representation Transfer from Large Language Models to End-to-End ASR Systemscited by 2
  26. 2024Traditional Machine Learning Models and Bidirectional Encoder Representations From Transformer (BERT)-Based Automatic Classification of Tweets About Eating Disorders: Algorithm Development and Validation Studycited by 2
  27. 2025Label-Context-Dependent Internal Language Model Estimation for CTCcited by 2
  28. 2022Multiple-hypothesis RNN-T Loss for Unsupervised Fine-tuning and Self-training of Neural Transducercited by 1
  29. 2022Partitioned Gradient Matching-based Data Subset Selection for Compute-Efficient Robust ASR Trainingcited by 1
  30. 2022Improving End-to-End Models for Set Prediction in Spoken Language Understandingcited by 0
  31. 2024Soft Language Identification for Language-Agnostic Many-to-One End-to-End Speech Translationcited by 0
  32. 2025Can Local Representation Alignment RNNs Solve Temporal Tasks?cited by 0