mlsanityData to 14 Oct 2026

Developing Real-time Streaming Transformer Transducer for Speech Recognition on Large-scale Dataset

arXiv 2010.11395 ↗First posted 22 Oct 2020cs.CL eess.AScited by 97

Not on the map. Why?

Abstract (arXiv)

Recently, Transformer based end-to-end models have achieved great success in many areas including speech recognition. However, compared to LSTM models, the heavy computational cost of the Transformer during inference is a key issue to prevent their applications. In this work, we explored the potential of Transformer Transducer (T-T) models for the fist pass decoding with low latency and fast speed on a large-scale dataset. We combine the idea of Transformer-XL and chunk-wise streaming processing to design a streamable Transformer Transducer model. We demonstrate that T-T outperforms the hybrid model, RNN Transducer (RNN-T), and streamable Transformer attention-based encoder-decoder model in the streaming scenario. Furthermore, the runtime cost and latency can be optimized with a relatively small look-ahead.

Key facts. Cited by 97 papers in the map (data through 14 Oct 2026). First posted on arXiv 22 Oct 2020.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2021: 16; 2022: 31; 2023: 27; 2024: 10; 2025: 6; 2026: 7.

97 papers.

Sort:

By area:

  1. 2021A Survey of Transformerscited by 401
  2. 2023Google USM: Scaling Automatic Speech Recognition Beyond 100 Languagescited by 231
  3. 2021Recent Advances in End-to-End Automatic Speech Recognitioncited by 140
  4. 2023End-to-End Speech Recognition: A Surveycited by 129
  5. 2023VioLA: Unified Codec Language Models for Speech Recognition, Synthesis, and Translationcited by 87
  6. 2021Keyword Transformer: A Self-Attention Model for Keyword Spottingcited by 60
  7. 2023Transformers in Speech Processing: A Surveycited by 39
  8. 2022Streaming Multi-Talker ASR with Token-Level Serialized Output Trainingcited by 37
  9. 2021Factorized Neural Transducer for Efficient Language Model Adaptationcited by 32
  10. 2022Large-Scale Streaming End-to-End Speech Translation with Neural Transducerscited by 31
  11. 2021A Configurable Multilingual Model is All You Need to Recognize All Languagescited by 28
  12. 2022GateHUB: Gated History Unit with Background Suppression for Online Action Detectioncited by 27
38 more
  1. 2023Stateful Conformer with Cache-based Inference for Streaming Automatic Speech Recognitioncited by 25
  2. 2022Real-Time Target Sound Extractioncited by 22
  3. 2023Hybrid Transducer and Attention based Encoder-Decoder Modeling for Speech-to-Text Taskscited by 20
  4. 2021On Addressing Practical Challenges for RNN-Transducercited by 19
  5. 2021Minimum Word Error Rate Training with Language Model Fusion for End-to-End Speech Recognitioncited by 19
  6. 2021Internal Language Model Adaptation with Text-Only Data for End-to-End Speech Recognitioncited by 17
  7. 2022Towards Contextual Spelling Correction for Customization of End-to-end Speech Recognition Systemscited by 17
  8. 2022Streaming Speaker-Attributed ASR with Token-Level Speaker Embeddingscited by 14
  9. 2022Variable Attention Masking for Configurable Transformer Transducer Speech Recognitioncited by 14
  10. 2022Bayes risk CTC: Controllable CTC alignment in Sequence-to-Sequence taskscited by 13
  11. 2022A Weakly-Supervised Streaming Multilingual Speech Model with Truly Zero-Shot Capabilitycited by 12
  12. 2021A Light-weight contextual spelling correction model for customizing transducer-based speech recognition systemscited by 11
  13. 2022Improving the fusion of acoustic and text representations in RNN-Tcited by 11
  14. 2022Fast and accurate factorized neural transducer for text adaption of end-to-end speech recognition modelscited by 11
  15. 2022Conformer with dual-mode chunked attention for joint online and offline ASRcited by 10
  16. 2022Improving Deliberation by Text-Only and Semi-Supervised Trainingcited by 10
  17. 2022Simulating realistic speech overlaps improves multi-talker ASRcited by 10
  18. 2022LAMASSU: Streaming Language-Agnostic Multilingual Speech Recognition and Translation Using Neural Transducerscited by 10
  19. 2022Points to Patches: Enabling the Use of Self-Attention for 3D Shape Recognitioncited by 9
  20. 2022A Language Agnostic Multilingual Streaming On-Device ASR Systemcited by 9
  21. 2023Improving Contextual Spelling Correction by External Acoustics Attention and Semantic Aware Data Augmentationcited by 9
  22. 2021Multi-View Self-Attention Based Transformer for Speaker Recognitioncited by 8
  23. 2022Minimum Latency Training of Sequence Transducers for Streaming End-to-End Speech Recognitioncited by 8
  24. 2023Approximate Nearest Neighbour Phrase Mining for Contextual Speech Recognitioncited by 8
  25. 2021SimulLR: Simultaneous Lip Reading Transducer with Attention-Guided Adaptive Memorycited by 7
  26. 2021Have best of both worlds: two-pass hybrid and E2E cascading framework for speech recognitioncited by 7
  27. 2022VarArray Meets t-SOT: Advancing the State of the Art of Streaming Distant Conversational Speech Recognitioncited by 7
  28. 2023DistillW2V2: A Small and Streaming Wav2vec 2.0 Based ASR Modelcited by 7
  29. 2023Chunked Attention-based Encoder-Decoder Model for Streaming Speech Recognitioncited by 7
  30. 2022LongFNT: Long-form Speech Recognition with Factorized Neural Transducercited by 6
  31. 2023Towards Effective and Compact Contextual Representation for Conformer Transducer Speech Recognition Systemscited by 6
  32. 2023Token-Level Serialized Output Training for Joint Streaming ASR and ST Leveraging Textual Alignmentscited by 6
  33. 2024Exploring SSL Discrete Tokens for Multilingual ASRcited by 6
  34. 2026Voxtral Realtimecited by 6
  35. 2021Streaming Transformer Transducer Based Speech Recognition Using Non-Causal Convolutioncited by 5
  36. 2022Learning a Dual-Mode Speech Recognition Model via Self-Pruningcited by 5
  37. 2022Learning from Data Streams: An Overview and Updatecited by 5
  38. 2023Online Transformers with Spiking Neurons for Fast Prosthetic Hand Controlcited by 5

50 most cited of 97 All 97 (JSON)