mlsanityData to 7 Oct 2026

Fast and parallel decoding for transducer

arXiv 2211.00484 ↗First posted 31 Oct 2022eess.AS cs.CL cs.LGcited by 12

On the map: Efficient LLM inference and serving → Speculative decoding

Abstract (arXiv)

The transducer architecture is becoming increasingly popular in the field of speech recognition, because it is naturally streaming as well as high in accuracy. One of the drawbacks of transducer is that it is difficult to decode in a fast and parallel way due to an unconstrained number of symbols that can be emitted per time step. In this work, we introduce a constrained version of transducer loss to learn strictly monotonic alignments between the sequences; we also improve the standard greedy search and beam search algorithms by limiting the number of symbols that can be emitted per time step in transducer decoding, making it more efficient to decode in parallel with batches. Furthermore, we propose an finite state automaton-based (FSA) parallel beam search algorithm that can run with graphs on GPU efficiently. The experiment results show that we achieve slight word error rate (WER) improvement as well as significant speedup in decoding. Our work is open-sourced and publicly availablefootnotehttps://github.com/k2-fsa/icefall.

Key facts. Cited by 12 papers in the map (data through 7 Oct 2026). Placed in Speculative decoding, one of 12 directions in Efficient LLM inference and serving. Builds on 12 of its 20 references that are in the map. First posted on arXiv 31 Oct 2022.

Builds on

12 of 20 references.

References · 12

  1. 2017Attention Is All You Needcited by 59,932
  2. 2018SentencePiece: A simple and language independent subword tokenizer and detokenizer for Neural Text Processingcited by 1,973
  3. 2020Conformer: Convolution-augmented Transformer for Speech Recognitioncited by 1,838
  4. 2019SpecAugment: A Simple Data Augmentation Method for Automatic Speech Recognitioncited by 1,355
  5. 2018ESPnet: End-to-End Speech Processing Toolkitcited by 554
  6. 2015MUSAN: A Music, Speech, and Noise Corpuscited by 400
  7. 2018Streaming End-to-end Speech Recognition For Mobile Devicescited by 230
  8. 2020A Streaming On-Device End-to-End Model Surpassing Server-Side Conventional Model Quality and Latencycited by 99
  9. 2022Pruned RNN-T for fast, memory-efficient ASR trainingcited by 46
  10. 2021Lhotse: a speech data representation library for the modern deep learning ecosystemcited by 33
  11. 2020Less Is More: Improved RNN-T Decoding Using Limited Label Context and Path Mergingcited by 26
  12. 2019RNN-T For Latency Controlled ASR With Improved Beam Searchcited by 19

Built on by

Cited by year

Citing papers per year: 2022: 1; 2023: 3; 2024: 5; 2025: 3.

12 papers.

  1. 2023FunASR: A Fundamental End-to-End Speech Recognition Toolkitcited by 92
  2. 2023Zipformer: A faster and better encoder for automatic speech recognitioncited by 77
  3. 2024Open-Source Conversational AI with SpeechBrain 1.0cited by 55
  4. 2024CR-CTC: Consistency regularization on CTC for improved speech recognitioncited by 15
  5. 2022Multi-blank Transducers for Speech Recognitioncited by 11
  6. 2024Label-Looping: Highly Efficient Decoding for Transducerscited by 9
  7. 2023GPU-Accelerated WFST Beam Search Decoder for CTC-based Speech Recognitioncited by 4
  8. 2025Pushing the Limits of Beam Search Decoding for Transducer-based ASR modelscited by 4
  9. 2025VietASR: Achieving Industry-level Vietnamese ASR with 50-hour labeled data and Large-Scale Speech Pretrainingcited by 3
  10. 2024HAINAN: Fast and Accurate Transducer for Hybrid-Autoregressive ASRcited by 2
  11. 2024FeruzaSpeech: A 60 Hour Uzbek Read Speech Corpus with Punctuation, Casing, and Contextcited by 0
  12. 2025Graph Connectionist Temporal Classification for Phoneme Recognitioncited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2022Blank Collapse: Compressing CTC emission for the faster decoding
  2. 2023Accelerating Transducers through Adjacent Token Merging
  3. 2024Parallel Decoding via Hidden Transfer for Lossless Large Language Model Acceleration
  4. 2021Equivalence of Segmental and Neural Transducer Modeling: A Proof of Concept
  5. 2026TRADE: Transducer-Augmented Decoder for Speech LLM
  6. 2023Fast Chain-of-Thought: A Glance of Future from Parallel Decoding Leads to Answers Faster
  7. 2023Matcha-TTS: A fast TTS architecture with conditional flow matching
  8. 2024Hardware-Aware Parallel Prompt Decoding for Memory-Efficient Acceleration of LLM Inference