mlsanityData to 14 Oct 2026

Pruned RNN-T for fast, memory-efficient ASR training

arXiv 2206.13236 ↗First posted 23 Jun 2022eess.AS cs.AI cs.LGcited by 46

Not on the map. Why?

Abstract (arXiv)

The RNN-Transducer (RNN-T) framework for speech recognition has been growing in popularity, particularly for deployed real-time ASR systems, because it combines high accuracy with naturally streaming recognition. One of the drawbacks of RNN-T is that its loss function is relatively slow to compute, and can use a lot of memory. Excessive GPU memory usage can make it impractical to use RNN-T loss in cases where the vocabulary size is large: for example, for Chinese character-based ASR. We introduce a method for faster and more memory-efficient RNN-T loss computation. We first obtain pruning bounds for the RNN-T recursion using a simple joiner network that is linear in the encoder and decoder embeddings; we can evaluate this without using much memory. We then use those pruning bounds to evaluate the full, non-linear joiner network.

Key facts. Cited by 46 papers in the map (data through 14 Oct 2026). Builds on 11 of its 17 references that are in the map. First posted on arXiv 23 Jun 2022.

Builds on

11 of 17 references.

References · 11

  1. 2017Attention Is All You Needcited by 60,361
  2. 2015Neural Machine Translation of Rare Words with Subword Unitscited by 2,918
  3. 2018SentencePiece: A simple and language independent subword tokenizer and detokenizer for Neural Text Processingcited by 1,989
  4. 2020Conformer: Convolution-augmented Transformer for Speech Recognitioncited by 1,863
  5. 2019SpecAugment: A Simple Data Augmentation Method for Automatic Speech Recognitioncited by 1,380
  6. 2021SpeechBrain: A General-Purpose Speech Toolkitcited by 406
  7. 2018Streaming End-to-end Speech Recognition For Mobile Devicescited by 234
  8. 2021TorchAudio: Building Blocks for Audio and Speech Processingcited by 83
  9. 2019Improving RNN Transducer Modeling for End-to-End Speech Recognitioncited by 68
  10. 2021Tied & Reduced RNN-T Decodercited by 36
  11. 2020Benchmarking LF-MMI, CTC and RNN-T Criteria for Streaming ASRcited by 11

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2022: 8; 2023: 10; 2024: 15; 2025: 9; 2026: 4.

46 papers.

Sort:

By area:

  1. 2024An Embarrassingly Simple Approach for LLM with Strong ASR Capacitycited by 88
  2. 2023Zipformer: A faster and better encoder for automatic speech recognitioncited by 80
  3. 2024GigaSpeech 2: An Evolving, Large-Scale and Multi-domain ASR Corpus for Low-Resource Languages with Automated Crawling, Transcription and Refinementcited by 34
  4. 2023PromptASR for contextualized ASR with controllable stylecited by 19
  5. 2025ZIPA: A family of efficient models for multilingual phone recognitioncited by 19
  6. 2022Accelerating RNN-T Training and Inference Using CTC guidancecited by 17
  7. 2024CR-CTC: Consistency regularization on CTC for improved speech recognitioncited by 17
  8. 2022Bayes risk CTC: Controllable CTC alignment in Sequence-to-Sequence taskscited by 13
  9. 2022Fast and parallel decoding for transducercited by 12
  10. 2022Multi-blank Transducers for Speech Recognitioncited by 11
  11. 2023Updated Corpora and Benchmarks for Long-Form Speech Recognitioncited by 10
  12. 2023Blank-regularized CTC for Frame Skipping in Neural Transducercited by 8
34 more
  1. 2022Delay-penalized transducer for low-latency streaming ASRcited by 7
  2. 2023Powerful and Extensible WFST Framework for RNN-Transducer Lossescited by 7
  3. 2023Transduce and Speak: Neural Transducer for Text-to-Speech with Semantic Token Predictioncited by 7
  4. 2022Predicting Multi-Codebook Vector Quantization Indexes for Knowledge Distillationcited by 6
  5. 2023CIF-T: A Novel CIF-based Transducer Architecture for Automatic Speech Recognitioncited by 6
  6. 2022Optimizing Bilingual Neural Transducer with Synthetic Code-switching Text Generationcited by 5
  7. 2024SpeechColab Leaderboard: An Open-Source Platform for Automatic Speech Recognition Evaluationcited by 4
  8. 2024Joint Beam Search Integrating CTC, Attention, and Transducer Decoderscited by 4
  9. 2025VietASR: Achieving Industry-level Vietnamese ASR with 50-hour labeled data and Large-Scale Speech Pretrainingcited by 3
  10. 2024TokenVerse: Towards Unifying Speech and NLP Tasks via Transducer-based ASRcited by 2
  11. 2024Clean Label Attacks against SLU Systemscited by 2
  12. 2025Early Attentive Sparsification Accelerates Neural Speech Transcriptioncited by 2
  13. 2022Neural Transducer Training: Reduced Memory Consumption with Sample-wise Computationcited by 1
  14. 2023Bayes Risk Transducer: Transducer with Controllable Alignment Predictioncited by 1
  15. 2024Utilizing Neural Transducers for Two-Stage Text-to-Speech via Semantic Token Predictioncited by 1
  16. 2024Self-consistent context aware conformer transducer for speech recognitioncited by 1
  17. 2024The X-LANCE Technical Report for Interspeech 2024 Speech Processing Using Discrete Speech Unit Challengecited by 1
  18. 2024Romanization Encoding For Multilingual ASRcited by 1
  19. 2024Fast Streaming Transducer ASR Prototyping via Knowledge Distillation with Whispercited by 1
  20. 2024Unifying Global and Near-Context Biasing in a Single Trie Passcited by 1
  21. 2024Lightweight Transducer Based on Frame-Level Criterioncited by 1
  22. 2024Transducer Consistency Regularization for Speech to Text Applicationscited by 1
  23. 2025HENT-SRT: Hierarchical Efficient Neural Transducer with Self-Distillation for Joint Speech Recognition and Translationcited by 1
  24. 2025Unifying Streaming and Non-streaming Zipformer-based ASRcited by 1
  25. 2025Enhanced Hybrid Transducer and Attention Encoder Decoder with Text Datacited by 1
  26. 2026Towards Comprehensive Semantic Speech Embeddings for Chinese Dialectscited by 1
  27. 2026Efficient Dialect-Aware Modeling and Conditioning for Low-Resource Taiwanese Hakka Speech Processingcited by 1
  28. 2023Globally Normalising the Transducer for Streaming Speech Recognitioncited by 0
  29. 2023RIR-SF: Room Impulse Response Based Spatial Feature for Target Speech Recognition in Multi-Channel Multi-Speaker Scenarioscited by 0
  30. 2025A Differentiable Alignment Framework for Sequence-to-Sequence Modeling via Optimal Transportcited by 0
  31. 2025TokenVerse++: Towards Flexible Multitask Learning with Dynamic Task Activationcited by 0
  32. 2025FLToP CTC: Frame-Level Token Pruning via Relative Threshold for Efficient and Memory-Saving Decoding on Diverse Platformscited by 0
  33. 2026SpatialEmb: Extract and Encode Spatial Information for 1-Stage Multi-channel Multi-speaker ASR on Arbitrary Microphone Arrayscited by 0
  34. 2026TRADE: Transducer-Augmented Decoder for Speech LLMcited by 0