mlsanityData to 7 Oct 2026

Label-Looping: Highly Efficient Decoding for Transducers

arXiv 2406.06220 ↗First posted 10 Jun 2024eess.AS cs.AI cs.CLcited by 9

Not on the map. Why?

Abstract (arXiv)

This paper introduces a highly efficient greedy decoding algorithm for Transducer-based speech recognition models. We redesign the standard nested-loop design for RNN-T decoding, swapping loops over frames and labels: the outer loop iterates over labels, while the inner loop iterates over frames searching for the next non-blank symbol. Additionally, we represent partial hypotheses in a special structure using CUDA tensors, supporting parallelized hypotheses manipulations. Experiments show that the label-looping algorithm is up to 2.0X faster than conventional batched decoding when using batch size 32. It can be further combined with other compiler or GPU call-related techniques to achieve even more speedup. Our algorithm is general-purpose and can work with both conventional Transducers and Token-and-Duration Transducers. We open-source our implementation to benefit the research community.

Key facts. Cited by 9 papers in the map (data through 7 Oct 2026). Builds on 16 of its 25 references that are in the map. First posted on arXiv 10 Jun 2024.

Builds on

16 of 25 references.

References · 16

  1. 2017Attention Is All You Needcited by 59,932
  2. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,160
  3. 2015Neural Machine Translation of Rare Words with Subword Unitscited by 2,898
  4. 2020Conformer: Convolution-augmented Transformer for Speech Recognitioncited by 1,838
  5. 2018ESPnet: End-to-End Speech Processing Toolkitcited by 554
  6. 2021SpeechBrain: A General-Purpose Speech Toolkitcited by 404
  7. 2019NeMo: a toolkit for building AI applications using Neural Modulescited by 230
  8. 2023End-to-End Speech Recognition: A Surveycited by 127
  9. 2023Fast Conformer with Linearly Scalable Attention for Efficient Speech Recognitioncited by 120
  10. 2023Efficient Sequence Transduction by Jointly Predicting Tokens and Durationscited by 56
  11. 2019Espresso: A Fast End-to-end Neural Speech Recognition Toolkitcited by 31
  12. 2020Accelerating RNN Transducer Inference via One-Step Constrained Beam Searchcited by 13
4 more
  1. 2022Fast and parallel decoding for transducercited by 12
  2. 2024Speed of Light Exact Greedy Decoding for RNN-T Speech Recognition Models on GPUcited by 12
  3. 2021FSR: Accelerating the Inference Process of Transducer-Based Models by Applying Fast-Skip Regularizationcited by 11
  4. 2022Multi-blank Transducers for Speech Recognitioncited by 11

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 1; 2025: 5; 2026: 3.

9 papers.

  1. 2025NGPU-LM: GPU-Accelerated N-Gram Language Model for Context-Biasing in Greedy ASR Decodingcited by 7
  2. 2025Pushing the Limits of Beam Search Decoding for Transducer-based ASR modelscited by 4
  3. 2025TurboBias: Universal ASR Context-Biasing powered by GPU-accelerated Phrase-Boosting Treecited by 4
  4. 2024HAINAN: Fast and Accurate Transducer for Hybrid-Autoregressive ASRcited by 2
  5. 2025TTS-Transducer: End-to-End Speech Synthesis with Neural Transducercited by 1
  6. 2025WIND: Accelerated RNN-T Decoding with Windowed Inference for Non-blank Detectioncited by 1
  7. 2026Reducing the Offline-Streaming Gap for Unified ASR Transducer with Consistency Regularizationcited by 1
  8. 2026Chunk-wise Attention Transducers for Fast and Accurate Streaming Speech-to-Textcited by 0
  9. 2026TurboBias 2.0: Streaming Context-Biasing for Production-Efficient ASR Systemscited by 0