mlsanityData to 7 Oct 2026

End-to-End Speech-to-Text Translation: A Survey

arXiv 2312.01053 ↗First posted 2 Dec 2023cs.CL cs.SD eess.AScited by 9

Not on the map. Why?

Abstract (arXiv)

Speech-to-text translation pertains to the task of converting speech signals in a language to text in another language. It finds its application in various domains, such as hands-free communication, dictation, video lecture transcription, and translation, to name a few. Automatic Speech Recognition (ASR), as well as Machine Translation(MT) models, play crucial roles in traditional ST translation, enabling the conversion of spoken language in its original form to written text and facilitating seamless cross-lingual communication. ASR recognizes spoken words, while MT translates the transcribed text into the target language. Such disintegrated models suffer from cascaded error propagation and high resource and training costs. As a result, researchers have been exploring end-to-end (E2E) models for ST translation. However, to our knowledge, there is no comprehensive review of existing works on E2E ST. The present survey, therefore, discusses the work in this direction. Our attempt has been to provide a comprehensive review of models employed, metrics, and datasets used for ST tasks, providing challenges and future research direction with new insights. We believe this review will be helpful to researchers working on various applications of ST models.

Key facts. Cited by 9 papers in the map (data through 7 Oct 2026). Builds on 71 of its 105 references that are in the map. First posted on arXiv 2 Dec 2023.

Builds on

71 of 105 references.

References · 71

  1. 2017Attention Is All You Needcited by 59,932
  2. 2015Distilling the Knowledge in a Neural Networkcited by 10,076
  3. 2019BERTScore: Evaluating Text Generation with BERTcited by 5,462
  4. 2019Parameter-Efficient Transfer Learning for NLPcited by 4,195
  5. 2020wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representationscited by 3,853
  6. 2019fairseq: A Fast, Extensible Toolkit for Sequence Modelingcited by 1,486
  7. 2020Multilingual Denoising Pre-training for Neural Machine Translationcited by 1,028
  8. 2019Libri-Light: A Benchmark for ASR with Limited or No Supervisioncited by 462
  9. 2021VoxPopuli: A Large-Scale Multilingual Speech Corpus for Representation Learning, Semi-Supervised Learning and Interpretationcited by 455
  10. 2021GigaSpeech: An Evolving, Multi-domain ASR Corpus with 10,000 Hours of Transcribed Audiocited by 391
  11. 2016Joint CTC-Attention based End-to-End Speech Recognition using Multi-task Learningcited by 254
  12. 2021Whitening Sentence Representations for Better Semantics and Faster Retrievalcited by 198
38 more
  1. 2020fairseq S2T: Fast Speech-to-Text Modeling with fairseqcited by 174
  2. 2018How2: A Large-scale Dataset for Multimodal Language Understandingcited by 137
  3. 2017Sequence-to-Sequence Models Can Directly Translate Foreign Speechcited by 121
  4. 2019Europarl-ST: A Multilingual Corpus For Speech Translation Of Parliamentary Debatescited by 114
  5. 2016Listen and Translate: A Proof of Concept for End-to-End Speech-to-Text Translationcited by 113
  6. 2021The Multilingual TEDx Corpus for Speech Recognition and Translationcited by 101
  7. 2020SimulEval: An Evaluation Toolkit for Simultaneous Translationcited by 94
  8. 2020ESPnet-ST: All-in-One Speech Translation Toolkitcited by 82
  9. 2019CIF: Continuous Integrate-and-Fire for End-to-End Speech Recognitioncited by 76
  10. 2018Pre-training on high-resource speech recognition improves low-resource speech-to-text translationcited by 70
  11. 2018End-to-End Automatic Speech Translation of Audiobookscited by 65
  12. 2022Unified Speech-Text Pre-training for Speech Translation and Recognitioncited by 64
  13. 2022Cross-modal Contrastive Learning for Speech Translationcited by 61
  14. 2019End-to-End Speech Translation with Knowledge Distillationcited by 58
  15. 2018Leveraging Weakly Supervised Data to Improve End-to-End Speech-to-Text Translationcited by 57
  16. 2020CoVoST: A Diverse Multilingual Speech-To-Text Translation Corpuscited by 57
  17. 2020Curriculum Pre-training for End-to-End Speech Translationcited by 57
  18. 2022STEMM: Self-learning with Speech-text Manifold Mixup for Speech Translationcited by 57
  19. 2020Bridging the Modality Gap for Speech-to-Text Translationcited by 55
  20. 2018Tied Multitask Learning for Neural Speech Translationcited by 54
  21. 2020Dual-decoder Transformer for Joint Automatic Speech Recognition and Multilingual Speech Translationcited by 54
  22. 2021Cascade versus Direct Speech Translation: Do the Differences Still Make a Difference?cited by 54
  23. 2021Lightweight Adapter Tuning for Multilingual Speech Translationcited by 53
  24. 2021Learning Shared Semantic Space for Speech-to-Text Translationcited by 49
  25. 2021End-to-end Speech Translation via Cross-modal Progressive Trainingcited by 47
  26. 2019A Comparative Study on End-to-end Speech to Text Translationcited by 40
  27. 2021RealTranS: End-to-End Simultaneous Speech Translation with Convolutional Weighted-Shrinking Transformercited by 39
  28. 2019Multilingual End-to-End Speech Translationcited by 37
  29. 2021Large-Scale Self- and Semi-Supervised Learning for Speech Translationcited by 36
  30. 2018Augmenting Librispeech with French Translations: A Multimodal Corpus for Direct Speech Translation Evaluationcited by 34
  31. 2022Attention as a Guide for Simultaneous Speech Translationcited by 32
  32. 2021Direct Simultaneous Speech-to-Text Translation Assisted by Synchronized Streaming ASRcited by 31
  33. 2022Large-Scale Streaming End-to-End Speech Translation with Neural Transducerscited by 31
  34. 2019Attention-Passing Models for Robust and Data-Efficient End-to-End Speech Translationcited by 30
  35. 2021Learning When to Translate for Streaming Speechcited by 30
  36. 2019Thinking Slow about Latency Evaluation for Simultaneous Machine Translationcited by 28
  37. 2022End-to-End Speech Translation for Code Switched Speechcited by 26
  38. 2022Revisiting End-to-End Speech-to-Text Translation From Scratchcited by 26

50 most cited of 71 All 71 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 2; 2025: 4; 2026: 3.

9 papers.

  1. 2024Recent Advances in End-to-End Simultaneous Speech Translationcited by 5
  2. 2024Towards Building Large Scale Datasets and State-of-the-Art Automatic Speech Translation Systems for 14 Indian Languagescited by 4
  3. 2025Locate-and-Focus: Enhancing Terminology Translation in Speech Language Modelscited by 3
  4. 2025TEDxTN: A Three-way Speech Translation Corpus for Code-Switched Tunisian Arabic - Englishcited by 3
  5. 2025Adaptive Inner Speech-Text Alignment for LLM-based Speech Translationcited by 2
  6. 2025POTSA: A Cross-Lingual Speech Alignment Framework for Speech-to-Text Translationcited by 0
  7. 2026Attention-guided Evidence Grounding for Spoken Question Answeringcited by 0
  8. 2026A Generalized Formalism of Auto-Regressive Decoding for Speech Processingcited by 0
  9. 2026Attention-Based Adaptive Policies for Simultaneous Speech-to-Text Translationcited by 0