mlsanityData to 7 Oct 2026

ESPnet-ST: All-in-One Speech Translation Toolkit

arXiv 2004.10234 ↗First posted 21 Apr 2020cs.CL cs.SD eess.AScited by 82

Not on the map. Why?

Abstract (arXiv)

We present ESPnet-ST, which is designed for the quick development of speech-to-speech translation systems in a single framework. ESPnet-ST is a new project inside end-to-end speech processing toolkit, ESPnet, which integrates or newly implements automatic speech recognition, machine translation, and text-to-speech functions for speech translation. We provide all-in-one recipes including data pre-processing, feature extraction, training, and decoding pipelines for a wide range of benchmark datasets. Our reproducible results can match or even outperform the current state-of-the-art performances; these pre-trained models are downloadable. The toolkit is publicly available at https://github.com/espnet/espnet.

Key facts. Cited by 82 papers in the map (data through 7 Oct 2026). First posted on arXiv 21 Apr 2020.

Builds on

No reference list yet.

Built on by

Cited by year

Citing papers per year: 2021: 26; 2022: 22; 2023: 21; 2024: 8; 2025: 5.

82 papers.

Sort:

By area:

  1. 2021SpeechT5: Unified-Modal Encoder-Decoder Pre-Training for Spoken Language Processingcited by 177
  2. 2021WaveFake: A Data Set to Facilitate Audio Deepfake Detectioncited by 78
  3. 2022Unified Speech-Text Pre-training for Speech Translation and Recognitioncited by 64
  4. 2022Cross-modal Contrastive Learning for Speech Translationcited by 61
  5. 2022STEMM: Self-learning with Speech-text Manifold Mixup for Speech Translationcited by 57
  6. 2021Cascade versus Direct Speech Translation: Do the Differences Still Make a Difference?cited by 54
  7. 2021Improving Speech Translation by Understanding and Learning from the Auxiliary Text Translation Taskcited by 54
  8. 2021Lightweight Adapter Tuning for Multilingual Speech Translationcited by 53
  9. 2021ESPnet-SLU: Advancing Spoken Language Understanding through ESPnetcited by 53
  10. 2021Learning Shared Semantic Space for Speech-to-Text Translationcited by 49
  11. 2021Stacked Acoustic-and-Textual Encoding: Integrating the Pre-trained Models into Speech Translation Encoderscited by 48
  12. 2021Fused Acoustic and Text Encoding for Multimodal Bilingual Pretraining and Speech Translationcited by 47
38 more
  1. 2021End-to-end Speech Translation via Cross-modal Progressive Trainingcited by 47
  2. 2021CTC-based Compression for Direct Speech Translationcited by 45
  3. 2022A3T: Alignment-Aware Acoustic and Text Pretraining for Speech Synthesis and Editingcited by 40
  4. 2022Information-Transport-based Policy for Simultaneous Translationcited by 40
  5. 2021RealTranS: End-to-End Simultaneous Speech Translation with Convolutional Weighted-Shrinking Transformercited by 39
  6. 2021An Exploration of Self-Supervised Pretrained Representations for End-to-End Speech Recognitioncited by 39
  7. 2023AlignAtt: Using Attention-based Audio-Translation Alignments as a Guide for Simultaneous Speech Translationcited by 35
  8. 2021Learning When to Translate for Streaming Speechcited by 30
  9. 2021Source and Target Bidirectional Knowledge Distillation for End-to-end Speech Translationcited by 27
  10. 2022Wav2Seq: Pre-training Speech-to-Text Encoder-Decoder Models Using Pseudo Languagescited by 26
  11. 2022Revisiting End-to-End Speech-to-Text Translation From Scratchcited by 26
  12. 2022CTC Alignments Improve Autoregressive Translationcited by 25
  13. 2023Pre-training for Speech Translation: CTC Meets Optimal Transportcited by 25
  14. 2021Searchable Hidden Intermediates for End-to-End Models of Decomposable Sequence Taskscited by 24
  15. 2022Does Simultaneous Speech Translation need Simultaneous Models?cited by 22
  16. 2023CMOT: Cross-modal Mixup via Optimal Transport for Speech Translationcited by 22
  17. 2022GigaST: A 10,000-hour Pseudo Speech Translation Corpuscited by 20
  18. 2023Hybrid Transducer and Attention based Encoder-Decoder Modeling for Speech-to-Text Taskscited by 20
  19. 2024Making LLMs Better Many-to-Many Speech-to-Text Translators with Curriculum Learningcited by 20
  20. 2023DUB: Discrete Unit Back-translation for Speech Translationcited by 18
  21. 2021BSTC: A Large-Scale Chinese-English Speech Translation Datasetcited by 16
  22. 2021Regularizing End-to-End Speech Translation with Triangular Decomposition Agreementcited by 16
  23. 2021Speechformer: Reducing Information Loss in Direct Speech Translationcited by 15
  24. 2022A Study of Gender Impact in Self-supervised Models for Speech-to-Text Systemscited by 14
  25. 2023When Good and Reproducible Results are a Giant with Feet of Clay: The Importance of Software Quality in NLPcited by 14
  26. 2023A Comparative Study on E-Branchformer vs Conformer in Speech Recognition, Translation, and Understanding Taskscited by 13
  27. 2022Leveraging Pseudo-labeled Data to Improve Direct Speech-to-Speech Translationcited by 12
  28. 2022Blockwise Streaming Transformer for Spoken Language Understanding and Simultaneous Speech Translationcited by 11
  29. 2022Resource-Efficient Separation Transformercited by 11
  30. 2023ESPnet-ST-v2: Multipurpose Spoken Language Translation Toolkitcited by 10
  31. 2021Non-autoregressive End-to-end Speech Translation with Parallel Autoregressive Rescoringcited by 9
  32. 2021Fast-MD: Fast Multi-Decoder End-to-End Speech Translation with Non-Autoregressive Hidden Intermediatescited by 9
  33. 2021Attention-based Multi-hypothesis Fusion for Speech Summarizationcited by 9
  34. 2023CTC-based Non-autoregressive Speech Translationcited by 9
  35. 2023End-to-End Speech-to-Text Translation: A Surveycited by 9
  36. 2023Incremental Blockwise Beam Search for Simultaneous Speech Translation with Controllable Quality-Latency Tradeoffcited by 8
  37. 2023Efficient Monotonic Multihead Attentioncited by 8
  38. 2021SpliceOut: A Simple and Efficient Audio Augmentation Methodcited by 7

50 most cited of 82 All 82 (JSON)