mlsanityData to 14 Oct 2026

A Comparative Study on End-to-end Speech to Text Translation

arXiv 1911.08870 ↗First posted 20 Nov 2019cs.CL cs.LG eess.AScited by 40

Not on the map. Why?

Abstract (arXiv)

Recent advances in deep learning show that end-to-end speech to text translation model is a promising approach to direct the speech translation field. In this work, we provide an overview of different end-to-end architectures, as well as the usage of an auxiliary connectionist temporal classification (CTC) loss for better convergence. We also investigate on pre-training variants such as initializing different components of a model using pre-trained models, and their impact on the final performance, which gives boosts up to 4% in BLEU and 5% in TER. Our experiments are performed on 270h IWSLT TED-talks En->De, and 100h LibriSpeech Audiobooks En->Fr. We also show improvements over the current end-to-end state-of-the-art systems on both tasks.

Key facts. Cited by 40 papers in the map (data through 14 Oct 2026). First posted on arXiv 20 Nov 2019.

Builds on

No reference list yet.

Built on by

Cited by year

Citing papers per year: 2021: 13; 2022: 9; 2023: 10; 2024: 4; 2025: 4.

40 papers.

Sort:

By area:

  1. 2021Direct speech-to-speech translation with discrete unitscited by 149
  2. 2022STEMM: Self-learning with Speech-text Manifold Mixup for Speech Translationcited by 57
  3. 2024A Comprehensive Review of Multimodal Large Language Models: Performance and Challenges Across Different Taskscited by 57
  4. 2021Cascade versus Direct Speech Translation: Do the Differences Still Make a Difference?cited by 54
  5. 2021Improving Speech Translation by Understanding and Learning from the Auxiliary Text Translation Taskcited by 54
  6. 2021Learning Shared Semantic Space for Speech-to-Text Translationcited by 49
  7. 2021Stacked Acoustic-and-Textual Encoding: Integrating the Pre-trained Models into Speech Translation Encoderscited by 48
  8. 2021CTC-based Compression for Direct Speech Translationcited by 45
  9. 2022Enhanced Direct Speech-to-Speech Translation Using Self-supervised Pre-training and Data Augmentationcited by 44
  10. 2021Source and Target Bidirectional Knowledge Distillation for End-to-end Speech Translationcited by 28
  11. 2022Revisiting End-to-End Speech-to-Text Translation From Scratchcited by 26
  12. 2023Pre-training for Speech Translation: CTC Meets Optimal Transportcited by 25
28 more
  1. 2024How "Real" is Your Real-Time Simultaneous Speech-to-Text Translation System?cited by 17
  2. 2021Speechformer: Reducing Information Loss in Direct Speech Translationcited by 15
  3. 2023Understanding and Bridging the Modality Gap for Speech Translationcited by 15
  4. 2022Efficient yet Competitive Speech Translation: FBK@IWSLT2022cited by 14
  5. 2022Improving Deliberation by Text-Only and Semi-Supervised Trainingcited by 10
  6. 2021Non-autoregressive End-to-end Speech Translation with Parallel Autoregressive Rescoringcited by 9
  7. 2021Fast-MD: Fast Multi-Decoder End-to-End Speech Translation with Non-Autoregressive Hidden Intermediatescited by 9
  8. 2023ComSL: A Composite Speech-Language Model for End-to-End Speech-to-Text Translationcited by 9
  9. 2023End-to-End Speech-to-Text Translation: A Surveycited by 9
  10. 2021Between Flexibility and Consistency: Joint Generation of Captions and Subtitlescited by 7
  11. 2022Direct Speech Translation for Automatic Subtitlingcited by 7
  12. 2021Dealing with training and test segmentation mismatch: FBK@IWSLT2021cited by 6
  13. 2023Rethinking and Improving Multi-task Learning for End-to-end Speech Translationcited by 6
  14. 2022Joint Speech Translation and Named Entity Recognitioncited by 5
  15. 2022Align, Write, Re-order: Explainable End-to-End Speech Translation via Operation Sequence Generationcited by 4
  16. 2025A Multimodal Symphony: Integrating Taste and Sound through Generative AIcited by 4
  17. 2023Bridging the Granularity Gap for Acoustic Modelingcited by 2
  18. 2024A Case Study on Filtering for End-to-End Speech Translationcited by 2
  19. 2024LLM-Ref: Enhancing Reference Handling in Technical Writing with Large Language Modelscited by 2
  20. 2021The NiuTrans End-to-End Speech Translation System for IWSLT 2021 Offline Taskcited by 1
  21. 2023Efficient CTC Regularization via Coarse Labels for End-to-End Speech Translationcited by 1
  22. 2023Improving End-to-End Speech Translation by Imitation-Based Knowledge Distillation with Synthetic Transcriptscited by 1
  23. 2025The Warmup Dilemma: How Learning Rate Strategies Impact Speech-to-Text Model Convergencecited by 1
  24. 2022Can one hear the position of nodes?cited by 0
  25. 2023Performance Comparison of Pre-trained Models for Speech-to-Text in Turkish: Whisper-Small and Wav2Vec2-XLS-R-300Mcited by 0
  26. 2023Bridging the Gaps of Both Modality and Language: Synchronous Bilingual CTC for Speech Translation and Speech Recognitioncited by 0
  27. 2025Overcoming Latency Bottlenecks in On-Device Speech Translation: A Cascaded Approach with Alignment-Based Streaming MTcited by 0
  28. 2025RosettaSpeech: Zero-Shot Speech-to-Speech Translation without Parallel Speechcited by 0