mlsanityData to 7 Oct 2026

Powerful and Extensible WFST Framework for RNN-Transducer Losses

arXiv 2303.10384 ↗First posted 18 Mar 2023eess.AS cs.AI cs.CLcited by 7

Not on the map. Why?

Abstract (arXiv)

This paper presents a framework based on Weighted Finite-State Transducers (WFST) to simplify the development of modifications for RNN-Transducer (RNN-T) loss. Existing implementations of RNN-T use CUDA-related code, which is hard to extend and debug. WFSTs are easy to construct and extend, and allow debugging through visualization. We introduce two WFST-powered RNN-T implementations: (1) "Compose-Transducer", based on a composition of the WFST graphs from acoustic and textual schema -- computationally competitive and easy to modify; (2) "Grid-Transducer", which constructs the lattice directly for further computations -- most compact, and computationally efficient. We illustrate the ease of extensibility through introduction of a new W-Transducer loss -- the adaptation of the Connectionist Temporal Classification with Wild Cards. W-Transducer (W-RNNT) consistently outperforms the standard RNN-T in a weakly-supervised data setup with missing parts of transcriptions at the beginning and end of utterances. All RNN-T losses are implemented with the k2 framework and are available in the NeMo toolkit.

Key facts. Cited by 7 papers in the map (data through 7 Oct 2026). Builds on 20 of its 32 references that are in the map. First posted on arXiv 18 Mar 2023.

Builds on

20 of 32 references.

References · 20

  1. 2016Layer Normalizationcited by 5,579
  2. 2020Conformer: Convolution-augmented Transformer for Speech Recognitioncited by 1,838
  3. 2019NeMo: a toolkit for building AI applications using Neural Modulescited by 230
  4. 2020Hybrid Autoregressive Transducer (hat)cited by 106
  5. 2015EESEN: End-to-End Speech Recognition using Deep RNN Models and WFST-based Decodingcited by 73
  6. 2020FastEmit: Low-latency Streaming ASR with Sequence-level Emission Regularizationcited by 66
  7. 2022Pruned RNN-T for fast, memory-efficient ASR trainingcited by 46
  8. 2020Alignment Restricted Streaming Recurrent Neural Network Transducercited by 42
  9. 2020Efficient minimum word error rate training of RNN-Transducer for end-to-end speech recognitioncited by 34
  10. 2019Minimum Bayes Risk Training of RNN-Transducer for End-to-End Speech Recognitioncited by 23
  11. 2020Differentiable Weighted Finite-State Transducerscited by 22
  12. 2020Semi-Supervised Speech Recognition via Graph-based Temporal Classificationcited by 19
8 more
  1. 2021CTC Variations Through New WFST Topologiescited by 17
  2. 2021Reducing Streaming ASR Model Delay with Self Alignmentcited by 16
  3. 2021FSR: Accelerating the Inference Process of Transducer-Based Models by Applying Fast-Skip Regularizationcited by 11
  4. 2022Star Temporal Classification: Sequence Classification with Partially Labeled Datacited by 10
  5. 2022Global Normalization for Streaming Speech Recognition in a Modular Frameworkcited by 10
  6. 2022Minimum Latency Training of Sequence Transducers for Streaming End-to-End Speech Recognitioncited by 8
  7. 2022Federated Domain Adaptation for ASR with Full Self-Supervisioncited by 6
  8. 2021Sequence Transduction with Graph-based Supervisioncited by 4

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 1; 2024: 0; 2025: 3; 2026: 3.

7 papers.

  1. 2023Align With Purpose: Optimize Desired Properties in CTC Models with a General Plug-and-Play Frameworkcited by 2
  2. 2025TTS-Transducer: End-to-End Speech Synthesis with Neural Transducercited by 1
  3. 2025RNN-Transducer-based Losses for Speech Recognition on Noisy Targetscited by 1
  4. 2025WST: Weakly Supervised Transducer for Automatic Speech Recognitioncited by 1
  5. 2026Modeling Overlapped Speech with Shufflescited by 0
  6. 2026Sometin Beta Pass Notin: Improving Multilingual ASR for Nigerian Languages via Knowledge Distillationcited by 0
  7. 2026A Training Criterion with Token-Level Tolerance to Transcription Ambiguity for Automatic Speech Recognitioncited by 0