mlsanityData to 7 Oct 2026

Developing RNN-T Models Surpassing High-Performance Hybrid Models with Customization Capability

arXiv 2007.15188 ↗First posted 30 Jul 2020eess.AS cs.CL cs.SDcited by 63

Not on the map. Why?

Abstract (arXiv)

Because of its streaming nature, recurrent neural network transducer (RNN-T) is a very promising end-to-end (E2E) model that may replace the popular hybrid model for automatic speech recognition. In this paper, we describe our recent development of RNN-T models with reduced GPU memory consumption during training, better initialization strategy, and advanced encoder modeling with future lookahead. When trained with Microsoft's 65 thousand hours of anonymized training data, the developed RNN-T model surpasses a very well trained hybrid model with both better recognition accuracy and lower latency. We further study how to customize RNN-T models to a new domain, which is important for deploying E2E models to practical scenarios. By comparing several methods leveraging text-only data in the new domain, we found that updating RNN-T's prediction and joint networks using text-to-speech generated from domain-specific text is the most effective.

Key facts. Cited by 63 papers in the map (data through 7 Oct 2026). First posted on arXiv 30 Jul 2020.

Builds on

No reference list yet.

Built on by

Cited by year

Citing papers per year: 2021: 21; 2022: 19; 2023: 14; 2024: 6; 2025: 3.

63 papers.

Sort:

By area:

  1. 2021Recent Advances in End-to-End Automatic Speech Recognitioncited by 138
  2. 2023End-to-End Speech Recognition: A Surveycited by 127
  3. 2021Internal Language Model Training for Domain-Adaptive End-to-End Speech Recognitioncited by 37
  4. 2021Factorized Neural Transducer for Efficient Language Model Adaptationcited by 31
  5. 2022Large-Scale Streaming End-to-End Speech Translation with Neural Transducerscited by 31
  6. 2021Transformer Based Deliberation for Two-Pass Speech Recognitioncited by 29
  7. 2021A Configurable Multilingual Model is All You Need to Recognize All Languagescited by 28
  8. 2021Investigating Methods to Improve Language Model Integration for Attention-based Encoder-Decoder ASR Modelscited by 25
  9. 2021Learning Word-Level Confidence For Subword End-to-End ASRcited by 22
  10. 2021Fast Text-Only Domain Adaptation of RNN-Transducer Prediction Networkcited by 19
  11. 2021On Addressing Practical Challenges for RNN-Transducercited by 19
  12. 2021Minimum Word Error Rate Training with Language Model Fusion for End-to-End Speech Recognitioncited by 19
38 more
  1. 2021Internal Language Model Adaptation with Text-Only Data for End-to-End Speech Recognitioncited by 17
  2. 2022Towards Contextual Spelling Correction for Customization of End-to-end Speech Recognition Systemscited by 17
  3. 2022Modular Hybrid Autoregressive Transducercited by 17
  4. 2021Dual Causal/Non-Causal Self-Attention for Streaming End-to-End Speech Recognitioncited by 14
  5. 2021A Light-weight contextual spelling correction model for customizing transducer-based speech recognition systemscited by 11
  6. 2022Improving the fusion of acoustic and text representations in RNN-Tcited by 11
  7. 2022Fast and accurate factorized neural transducer for text adaption of end-to-end speech recognition modelscited by 11
  8. 2023Text-only domain adaptation for end-to-end ASR using integrated text-to-mel-spectrogram generatorcited by 11
  9. 2022Boosting Cross-Domain Speech Recognition with Self-Supervisioncited by 10
  10. 2022Improving Deliberation by Text-Only and Semi-Supervised Trainingcited by 10
  11. 2021Residual Energy-Based Models for End-to-End Speech Recognitioncited by 9
  12. 2023JEIT: Joint End-to-End Model and Internal Language Model Training for Speech Recognitioncited by 9
  13. 2023Improving Contextual Spelling Correction by External Acoustics Attention and Semantic Aware Data Augmentationcited by 9
  14. 2021Comparing the Benefit of Synthetic Training Data for Various Automatic Speech Recognition Architecturescited by 8
  15. 2021Listen with Intent: Improving Speech Recognition with Audio-to-Intent Front-Endcited by 8
  16. 2021CoDERT: Distilling Encoder Representations with Co-learning for Transducer-based Speech Recognitioncited by 8
  17. 2022Minimum Latency Training of Sequence Transducers for Streaming End-to-End Speech Recognitioncited by 8
  18. 2021Improving Confidence Estimation on Out-of-Domain Data for End-to-End Speech Recognitioncited by 7
  19. 2022A Simple Baseline for Domain Adaptation in End to End ASR Systems Using Synthetic Datacited by 7
  20. 2021Multi-mode Transformer Transducer with Stochastic Future Contextcited by 6
  21. 2022Distribution Aware Metrics for Conditional Natural Language Generationcited by 6
  22. 2023CIF-T: A Novel CIF-based Transducer Architecture for Automatic Speech Recognitioncited by 6
  23. 2023Improving Large-scale Deep Biasing with Phoneme Features and Text-only Data in Streaming Transducercited by 6
  24. 2023Conformer-Based Speech Recognition On Extreme Edge-Computing Devicescited by 6
  25. 2022A Likelihood Ratio based Domain Adaptation Method for E2E Modelscited by 5
  26. 2022Joint Speech Recognition and Audio Captioningcited by 5
  27. 2022Adaptive Discounting of Implicit Language Models in RNN-Transducerscited by 5
  28. 2023Text-only Domain Adaptation using Unified Speech-Text Representation in Transducercited by 5
  29. 2023Optimizing Two-Pass Cross-Lingual Transfer Learning: Phoneme Recognition and Phoneme to Grapheme Translationcited by 5
  30. 2023Pseudo-Labeling for Domain-Agnostic Bangla Automatic Speech Recognitioncited by 4
  31. 2024Text Injection for Neural Contextual Biasingcited by 3
  32. 2021Dynamic Acoustic Unit Augmentation With BPE-Dropout for Low-Resource End-to-End Speech Recognitioncited by 2
  33. 2022An Empirical Study of Language Model Integration for Transducer based Speech Recognitioncited by 2
  34. 2022Accelerating Inference and Language Model Fusion of Recurrent Neural Network Transducers via End-to-End 4-bit Quantizationcited by 2
  35. 2023External Language Model Integration for Factorized Neural Transducerscited by 2
  36. 2023Speaker Mask Transformer for Multi-talker Overlapped Speech Recognitioncited by 2
  37. 2025Retrieval-Augmented Speech Recognition Approach for Domain Challengescited by 2
  38. 2021Integrating Categorical Features in End-to-End ASRcited by 1

50 most cited of 63 All 63 (JSON)