mlsanityData to 14 Oct 2026

EESEN: End-to-End Speech Recognition using Deep RNN Models and WFST-based Decoding

arXiv 1507.08240 ↗First posted 29 Jul 2015cs.CL cs.LGcited by 73

Not on the map. Why?

Abstract (arXiv)

The performance of automatic speech recognition (ASR) has improved tremendously due to the application of deep neural networks (DNNs). Despite this progress, building a new ASR system remains a challenging task, requiring various resources, multiple training stages and significant expertise. This paper presents our Eesen framework which drastically simplifies the existing pipeline to build state-of-the-art ASR systems. Acoustic modeling in Eesen involves learning a single recurrent neural network (RNN) predicting context-independent targets (phonemes or characters). To remove the need for pre-generated frame labels, we adopt the connectionist temporal classification (CTC) objective function to infer the alignments between speech and label sequences. A distinctive feature of Eesen is a generalized decoding approach based on weighted finite-state transducers (WFSTs), which enables the efficient incorporation of lexicons and language models into CTC decoding. Experiments show that compared with the standard hybrid DNN systems, Eesen achieves comparable word error rates (WERs), while at the same time speeding up decoding significantly.

Key facts. Cited by 73 papers in the map (data through 14 Oct 2026). First posted on arXiv 29 Jul 2015.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2021: 20; 2022: 22; 2023: 17; 2024: 8; 2025: 5; 2026: 1.

73 papers.

Sort:

By area:

  1. 2021AgentFormer: Agent-Aware Transformers for Socio-Temporal Multi-Agent Forecastingcited by 331
  2. 2021WenetSpeech: A 10000+ Hours Multi-domain Mandarin Corpus for Speech Recognitioncited by 197
  3. 2021Recent Advances in End-to-End Automatic Speech Recognitioncited by 140
  4. 2023End-to-End Speech Recognition: A Surveycited by 129
  5. 2023Hierarchically Gated Recurrent Neural Network for Sequence Modelingcited by 102
  6. 2021Intermediate Loss Regularization for CTC-based Speech Recognitioncited by 96
  7. 2021Why does CTC result in peaky behavior?cited by 33
  8. 2021Fast End-to-End Speech Recognition via Non-Autoregressive Models and Cross-Modal Knowledge Transferring from BERTcited by 31
  9. 2022Wav2Seq: Pre-training Speech-to-Text Encoder-Decoder Models Using Pseudo Languagescited by 26
  10. 2022ASR2K: Speech Recognition for Around 2000 Languages without Audiocited by 26
  11. 2021On Addressing Practical Challenges for RNN-Transducercited by 19
  12. 2021CTC Variations Through New WFST Topologiescited by 17
38 more
  1. 2024Whistle: Data-Efficient Multilingual and Crosslingual Speech Recognition via Weakly Phonetic Supervisioncited by 17
  2. 2023Accelerating Transducers through Adjacent Token Mergingcited by 15
  3. 2024Comparing Discrete and Continuous Space LLMs for Speech Recognitioncited by 15
  4. 2024Decoder-only Architecture for Streaming End-to-end Speech Recognitioncited by 13
  5. 2021Speech Emotion Recognition Using Quaternion Convolutional Neural Networkscited by 11
  6. 2021WNARS: WFST based Non-autoregressive Streaming End-to-End Speech Recognitioncited by 10
  7. 2021Differentiable Allophone Graphs for Language-Universal Speech Recognitioncited by 10
  8. 2022Star Temporal Classification: Sequence Classification with Partially Labeled Datacited by 10
  9. 2023Bypass Temporal Classification: Weakly Supervised Automatic Speech Recognition with Imperfect Transcriptscited by 10
  10. 2021Dynamic Encoder Transducer: A Flexible Solution For Trading Off Accuracy For Latencycited by 9
  11. 2021What do End-to-End Speech Models Learn about Speaker, Language and Channel Information? A Layer-wise and Neuron-level Analysiscited by 9
  12. 2022LegoNN: Building Modular Encoder-Decoder Modelscited by 9
  13. 2022Streaming parallel transducer beam search with fast-slow cascaded encoderscited by 8
  14. 2022On Compressing Sequences for Self-Supervised Speech Modelscited by 8
  15. 2023Blank-regularized CTC for Frame Skipping in Neural Transducercited by 8
  16. 2023Self-distillation Regularized Connectionist Temporal Classification Loss for Text Recognition: A Simple Yet Effective Approachcited by 8
  17. 2023Learning from Flawed Data: Weakly Supervised Automatic Speech Recognitioncited by 8
  18. 2021Multi-mode Transformer Transducer with Stochastic Future Contextcited by 7
  19. 2022Losses Can Be Blessings: Routing Self-Supervised Speech Representations Towards Efficient Multilingual and Multitask Speech Processingcited by 7
  20. 2023Powerful and Extensible WFST Framework for RNN-Transducer Lossescited by 7
  21. 2023Weakly-supervised forced alignment of disfluent speech using phoneme-level modelingcited by 7
  22. 2023Alternative Pseudo-Labeling for Semi-Supervised Automatic Speech Recognitioncited by 6
  23. 2023Personalization of CTC-based End-to-End Speech Recognition Using Pronunciation-Driven Subword Tokenizationcited by 6
  24. 2021A Unified Speaker Adaptation Approach for ASRcited by 5
  25. 2022LiteLSTM Architecture for Deep Recurrent Neural Networkscited by 5
  26. 2022Insights on Neural Representations for End-to-End Speech Recognitioncited by 5
  27. 2023Multimodal Data and Resource Efficient Device-Directed Speech Detection with Large Foundation Modelscited by 5
  28. 2024A Multimodal Approach to Device-Directed Speech Detection with Large Language Modelscited by 5
  29. 2021Cross-Modal Transformer-Based Neural Correction Models for Automatic Speech Recognitioncited by 4
  30. 2022Improving CTC-based ASR Models with Gated Interlayer Collaborationcited by 4
  31. 2025Code-Switching in End-to-End Automatic Speech Recognition: A Systematic Literature Reviewcited by 4
  32. 2021Arabic aspect based sentiment analysis using bidirectional GRU based modelscited by 3
  33. 2021End-to-End Rich Transcription-Style Automatic Speech Recognition with Semi-Supervised Learningcited by 3
  34. 2022Improving Automatic Speech Recognition for Non-Native English with Transfer Learning and Language Model Decodingcited by 3
  35. 2022Memory-Efficient Training of RNN-Transducer with Sampled Softmaxcited by 3
  36. 2022Supervised Attention in Sequence-to-Sequence Models for Speech Recognitioncited by 3
  37. 2022Blank Collapse: Compressing CTC emission for the faster decodingcited by 3
  38. 2024Transformer-based Model for ASR N-Best Rescoring and Rewritingcited by 3

50 most cited of 73 All 73 (JSON)