mlsanityData to 14 Oct 2026

ESPnet: End-to-End Speech Processing Toolkit

arXiv 1804.00015 ↗First posted 30 Mar 2018cs.CLcited by 558

Not on the map. Why?

Abstract (arXiv)

This paper introduces a new open source platform for end-to-end speech processing named ESPnet. ESPnet mainly focuses on end-to-end automatic speech recognition (ASR), and adopts widely-used dynamic neural network toolkits, Chainer and PyTorch, as a main deep learning engine. ESPnet also follows the Kaldi ASR toolkit style for data processing, feature extraction/format, and recipes to provide a complete setup for speech recognition and other speech processing experiments. This paper explains a major architecture of this software platform, several important functionalities, which differentiate ESPnet from other open source ASR toolkits, and experimental results with major ASR benchmarks.

Key facts. Cited by 558 papers in the map (data through 14 Oct 2026). First posted on arXiv 30 Mar 2018.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2021: 109; 2022: 137; 2023: 121; 2024: 102; 2025: 61; 2026: 28.

558 papers.

Sort:

By area:

  1. 2023Scaling Speech Technology to 1,000+ Languagescited by 411
  2. 2021SpeechBrain: A General-Purpose Speech Toolkitcited by 406
  3. 2021GigaSpeech: An Evolving, Multi-domain ASR Corpus with 10,000 Hours of Transcribed Audiocited by 399
  4. 2021WenetSpeech: A 10000+ Hours Multi-domain Mandarin Corpus for Speech Recognitioncited by 197
  5. 2021SpeechT5: Unified-Modal Encoder-Decoder Pre-Training for Spoken Language Processingcited by 180
  6. 2024GAMA: A Large Audio-Language Model with Advanced Audio Understanding and Complex Reasoning Abilitiescited by 156
  7. 2023StyleTTS 2: Towards Human-Level Text-to-Speech through Style Diffusion and Adversarial Training with Large Speech Language Modelscited by 152
  8. 2022Paraformer: Fast and Accurate Parallel Transformer for Non-autoregressive End-to-End Speech Recognitioncited by 144
  9. 2021Recent Advances in End-to-End Automatic Speech Recognitioncited by 140
  10. 2025Audio Flamingo 2: An Audio-Language Model with Long-Audio Understanding and Expert Reasoning Abilitiescited by 133
  11. 2021WeNet: Production oriented Streaming and Non-streaming End-to-End Speech Recognition Toolkitcited by 125
  12. 2021Intermediate Loss Regularization for CTC-based Speech Recognitioncited by 96
38 more
  1. 2022Visual Speech Recognition for Multiple Languages in the Wildcited by 95
  2. 2023FunASR: A Fundamental End-to-End Speech Recognition Toolkitcited by 95
  3. 2022E-Branchformer: Branchformer with Enhanced merging for speech recognitioncited by 92
  4. 2022Branchformer: Parallel MLP-Attention Architectures to Capture Local and Global Context for Speech Recognition and Understandingcited by 91
  5. 2023Zipformer: A faster and better encoder for automatic speech recognitioncited by 80
  6. 2021WaveFake: A Data Set to Facilitate Audio Deepfake Detectioncited by 78
  7. 2021UniSpeech: Unified Speech Representation Learning with Labeled and Unlabeled Datacited by 77
  8. 2024OWSM v3.1: Better and Faster Open Whisper-Style Speech Models based on E-Branchformercited by 74
  9. 2023ML-SUPERB: Multilingual Speech Universal PERformance Benchmarkcited by 71
  10. 2021Performance-Efficiency Trade-offs in Unsupervised Pre-training for Speech Recognitioncited by 68
  11. 2024mHuBERT-147: A Compact Multilingual HuBERT Modelcited by 66
  12. 2021Earnings-21: A Practical Benchmark for ASR in the Wildcited by 62
  13. 2021Relaxing the Conditional Independence Assumption of CTC-based ASR by Conditioning on Intermediate Predictionscited by 61
  14. 2024Mamba in Speech: Towards an Alternative to Self-Attentioncited by 60
  15. 2024YODAS: Youtube-Oriented Dataset for Audio and Speechcited by 60
  16. 2021StarGANv2-VC: A Diverse, Unsupervised, Non-parallel Framework for Natural-Sounding Voice Conversioncited by 58
  17. 2023Efficient Sequence Transduction by Jointly Predicting Tokens and Durationscited by 58
  18. 2023Can Generative Large Language Models Perform ASR Error Correction?cited by 57
  19. 2023Whispering LLaMA: A Cross-Modal Generative Error Correction Framework for Speech Recognitioncited by 56
  20. 2024Open-Source Conversational AI with SpeechBrain 1.0cited by 56
  21. 2021ESPnet-SLU: Advancing Spoken Language Understanding through ESPnetcited by 53
  22. 2024Towards Robust Speech Representation Learning for Thousands of Languagescited by 50
  23. 2021Stacked Acoustic-and-Textual Encoding: Integrating the Pre-trained Models into Speech Translation Encoderscited by 48
  24. 2022WeNet 2.0: More Productive End-to-End Speech Recognition Toolkitcited by 47
  25. 2022Jointly Learning Visual and Auditory Speech Representations from Raw Datacited by 47
  26. 2021FastCorrect: Fast Error Correction with Edit Alignment for Automatic Speech Recognitioncited by 43
  27. 2023Prompting the Hidden Talent of Web-Scale Speech Models for Zero-Shot Task Generalizationcited by 43
  28. 2023N-best T5: Robust ASR Error Correction using Multiple Input Hypotheses and Constrained Decoding Spacecited by 42
  29. 2023Reproducing Whisper-Style Training Using an Open-Source Toolkit and Publicly Available Datacited by 42
  30. 2023HyPoradise: An Open Baseline for Generative Speech Recognition with Large Language Modelscited by 42
  31. 2023Exploring Speech Recognition, Translation, and Understanding with Discrete Speech Units: A Comparative Studycited by 42
  32. 2021Momentum Pseudo-Labeling for Semi-Supervised Speech Recognitioncited by 40
  33. 2022Open Source MagicData-RAMC: A Rich Annotated Mandarin Conversational(RAMC) Speech Datasetcited by 40
  34. 2021An Exploration of Self-Supervised Pretrained Representations for End-to-End Speech Recognitioncited by 39
  35. 2021QASR: QCRI Aljazeera Speech Resource -- A Large Scale Annotated Arabic Speech Corpuscited by 38
  36. 2024OWSM-CTC: An Open Encoder-Only Speech Foundation Model for Speech Recognition, Translation, and Language Identificationcited by 37
  37. 2023Structural and Statistical Texture Knowledge Distillation for Semantic Segmentationcited by 36
  38. 2021Exploiting Adapters for Cross-lingual Low-resource Speech Recognitioncited by 35

50 most cited of 558 All 558 (JSON)