mlsanityData to 14 Oct 2026

Blank-regularized CTC for Frame Skipping in Neural Transducer

arXiv 2305.11558 ↗First posted 19 May 2023eess.AS cs.CLcited by 8

Not on the map. Why?

Abstract (arXiv)

Neural Transducer and connectionist temporal classification (CTC) are popular end-to-end automatic speech recognition systems. Due to their frame-synchronous design, blank symbols are introduced to address the length mismatch between acoustic frames and output tokens, which might bring redundant computation. Previous studies managed to accelerate the training and inference of neural Transducers by discarding frames based on the blank symbols predicted by a co-trained CTC. However, there is no guarantee that the co-trained CTC can maximize the ratio of blank symbols. This paper proposes two novel regularization methods to explicitly encourage more blanks by constraining the self-loop of non-blank symbols in the CTC. It is interesting to find that the frame reduction ratio of the neural Transducer can approach the theoretical boundary. Experiments on LibriSpeech corpus show that our proposed method accelerates the inference of neural Transducer by 4 times without sacrificing performance. Our work is open-sourced and publicly available https://github.com/k2-fsa/icefall.

Key facts. Cited by 8 papers in the map (data through 14 Oct 2026). Builds on 16 of its 24 references that are in the map. First posted on arXiv 19 May 2023.

Builds on

16 of 24 references.

References · 16

  1. 2015Neural Machine Translation of Rare Words with Subword Unitscited by 2,918
  2. 2020Conformer: Convolution-augmented Transformer for Speech Recognitioncited by 1,863
  3. 2019SpecAugment: A Simple Data Augmentation Method for Automatic Speech Recognitioncited by 1,380
  4. 2013Speech Recognition with Deep Recurrent Neural Networkscited by 669
  5. 2015Listen, Attend and Spellcited by 543
  6. 2015MUSAN: A Music, Speech, and Noise Corpuscited by 425
  7. 2017An analysis of incorporating an external language model into a sequence-to-sequence modelcited by 111
  8. 2016Towards better decoding and language model integration in sequence to sequence modelscited by 100
  9. 2015EESEN: End-to-End Speech Recognition using Deep RNN Models and WFST-based Decodingcited by 73
  10. 2022Pruned RNN-T for fast, memory-efficient ASR trainingcited by 46
  11. 2021Lhotse: a speech data representation library for the modern deep learning ecosystemcited by 33
  12. 2021Tiny Transducer: A Highly-efficient Speech Recognition Model on Edge Devicescited by 19
4 more
  1. 2021CTC Variations Through New WFST Topologiescited by 17
  2. 2022Accelerating RNN-T Training and Inference Using CTC guidancecited by 17
  3. 2021FSR: Accelerating the Inference Process of Transducer-Based Models by Applying Fast-Skip Regularizationcited by 11
  4. 2022An Empirical Study of Language Model Integration for Transducer based Speech Recognitioncited by 2

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 2; 2024: 2; 2025: 3; 2026: 1.

8 papers.

  1. 2023Advancing Test-Time Adaptation in Wild Acoustic Test Settingscited by 7
  2. 2023Key Frame Mechanism For Efficient Conformer Based End-to-end Speech Recognitioncited by 2
  3. 2024Skipformer: A Skip-and-Recover Strategy for Efficient Speech Recognitioncited by 2
  4. 2024Boosting Hybrid Autoregressive Transducer-based ASR with Internal Acoustic Model Training and Dual Blank Thresholdingcited by 1
  5. 2025WIND: Accelerated RNN-T Decoding with Windowed Inference for Non-blank Detectioncited by 1
  6. 2025A Differentiable Alignment Framework for Sequence-to-Sequence Modeling via Optimal Transportcited by 0
  7. 2025End-to-end Speech Recognition with similar length speech and textcited by 0
  8. 2026OpenFS: Multi-Hand-Capable Fingerspelling Recognition with Implicit Signing-Hand Detection and Frame-Wise Letter-Conditioned Synthesiscited by 0