mlsanityData to 14 Oct 2026

Multi-blank Transducers for Speech Recognition

arXiv 2211.03541 ↗First posted 4 Nov 2022eess.AS cs.LG cs.SDcited by 11

Not on the map. Why?

Abstract (arXiv)

This paper proposes a modification to RNN-Transducer (RNN-T) models for automatic speech recognition (ASR). In standard RNN-T, the emission of a blank symbol consumes exactly one input frame; in our proposed method, we introduce additional blank symbols, which consume two or more input frames when emitted. We refer to the added symbols as big blanks, and the method multi-blank RNN-T. For training multi-blank RNN-Ts, we propose a novel logit under-normalization method in order to prioritize emissions of big blanks. With experiments on multiple languages and datasets, we show that multi-blank RNN-T methods could bring relative speedups of over +90%/+139% to model inference for English Librispeech and German Multilingual Librispeech datasets, respectively. The multi-blank RNN-T method also improves ASR accuracy consistently. We will release our implementation of the method in the NeMo (https://github.com/NVIDIA/NeMo) toolkit.

Key facts. Cited by 11 papers in the map (data through 14 Oct 2026). Builds on 21 of its 32 references that are in the map. First posted on arXiv 4 Nov 2022.

Builds on

21 of 32 references.

References · 21

  1. 2015Neural Machine Translation of Rare Words with Subword Unitscited by 2,918
  2. 2020Conformer: Convolution-augmented Transformer for Speech Recognitioncited by 1,863
  3. 2019fairseq: A Fast, Extensible Toolkit for Sequence Modelingcited by 1,488
  4. 2018ESPnet: End-to-End Speech Processing Toolkitcited by 558
  5. 2015Listen, Attend and Spellcited by 543
  6. 2020MLS: A Large-Scale Multilingual Dataset for Speech Researchcited by 476
  7. 2021VoxPopuli: A Large-Scale Multilingual Speech Corpus for Representation Learning, Semi-Supervised Learning and Interpretationcited by 463
  8. 2021SpeechBrain: A General-Purpose Speech Toolkitcited by 406
  9. 2015Attention-Based Models for Speech Recognitioncited by 401
  10. 2019NeMo: a toolkit for building AI applications using Neural Modulescited by 235
  11. 2020Transformer Transducer: A Streamable Speech Recognition Model with Transformer Encoders and RNN-T Losscited by 235
  12. 2020ContextNet: Improving Convolutional Neural Networks for Automatic Speech Recognition with Global Contextcited by 97
9 more
  1. 2020FastEmit: Low-latency Streaming ASR with Sequence-level Emission Regularizationcited by 69
  2. 2019Improving RNN Transducer Modeling for End-to-End Speech Recognitioncited by 68
  3. 2022Pruned RNN-T for fast, memory-efficient ASR trainingcited by 46
  4. 2020Alignment Restricted Streaming Recurrent Neural Network Transducercited by 42
  5. 2019Espresso: A Fast End-to-end Neural Speech Recognition Toolkitcited by 31
  6. 2022Fast and parallel decoding for transducercited by 12
  7. 2022Star Temporal Classification: Sequence Classification with Partially Labeled Datacited by 10
  8. 2020Exploring Pre-training with Alignments for RNN Transducer based End-to-End Speech Recognitioncited by 9
  9. 2022ConvRNN-T: Convolutional Augmented Recurrent Neural Network Transducers for Streaming Speech Recognitioncited by 8

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 4; 2024: 4; 2025: 2; 2026: 1.

11 papers.

  1. 2023Efficient Sequence Transduction by Jointly Predicting Tokens and Durationscited by 58
  2. 2024Speed of Light Exact Greedy Decoding for RNN-T Speech Recognition Models on GPUcited by 12
  3. 2024Label-Looping: Highly Efficient Decoding for Transducerscited by 9
  4. 2023Token-Level Serialized Output Training for Joint Streaming ASR and ST Leveraging Textual Alignmentscited by 6
  5. 2023Key Frame Mechanism For Efficient Conformer Based End-to-end Speech Recognitioncited by 2
  6. 2024HAINAN: Fast and Accurate Transducer for Hybrid-Autoregressive ASRcited by 2
  7. 2023Bayes Risk Transducer: Transducer with Controllable Alignment Predictioncited by 1
  8. 2024Lightweight Transducer Based on Frame-Level Criterioncited by 1
  9. 2025WIND: Accelerated RNN-T Decoding with Windowed Inference for Non-blank Detectioncited by 1
  10. 2025A Study on Regularization-Based Continual Learning Methods for Indic ASRcited by 1
  11. 2026Chunk-wise Attention Transducers for Fast and Accurate Streaming Speech-to-Textcited by 0