mlsanityData to 14 Oct 2026

When End-to-End is Overkill: Rethinking Cascaded Speech-to-Text Translation

arXiv 2502.00377 ↗First posted 1 Feb 2025cs.CL cs.AI cs.MMcited by 4

Not on the map. Why?

Abstract (arXiv)

Though end-to-end speech-to-text translation has been a great success, we argue that the cascaded speech-to-text translation model still has its place, which is usually criticized for the error propagation between automatic speech recognition (ASR) and machine translation (MT) models. In this paper, we explore the benefits of incorporating multiple candidates from ASR and self-supervised speech features into MT. Our analysis reveals that the primary cause of cascading errors stems from the increased divergence between similar samples in the speech domain when mapped to the text domain. By including multiple candidates and self-supervised speech features, our approach allows the machine translation model to choose the right words and ensure precise translation using various speech samples. This strategy minimizes error spread and takes advantage of large ASR and MT datasets, along with pre-trained ASR/MT models, while addressing associated issues.

Key facts. Cited by 4 papers in the map (data through 14 Oct 2026). Builds on 26 of its 34 references that are in the map. First posted on arXiv 1 Feb 2025.

Builds on

26 of 34 references.

References · 26

  1. 2021On the Opportunities and Risks of Foundation Modelscited by 4,033
  2. 2021HuBERT: Self-Supervised Speech Representation Learning by Masked Prediction of Hidden Unitscited by 2,342
  3. 2020Multilingual Denoising Pre-training for Neural Machine Translationcited by 1,035
  4. 2021SUPERB: Speech processing Universal PERformance Benchmarkcited by 666
  5. 2021VoxPopuli: A Large-Scale Multilingual Speech Corpus for Representation Learning, Semi-Supervised Learning and Interpretationcited by 463
  6. 2021GigaSpeech: An Evolving, Multi-domain ASR Corpus with 10,000 Hours of Transcribed Audiocited by 399
  7. 2021Generative Spoken Language Modeling from Raw Audiocited by 340
  8. 2021Pre-Trained Models: Past, Present and Futurecited by 319
  9. 2021Speech Resynthesis from Discrete Disentangled Self-Supervised Representationscited by 243
  10. 2021Direct speech-to-speech translation with discrete unitscited by 149
  11. 2021WeNet: Production oriented Streaming and Non-streaming End-to-End Speech Recognition Toolkitcited by 125
  12. 2016Listen and Translate: A Proof of Concept for End-to-End Speech-to-Text Translationcited by 113
14 more
  1. 2018Leveraging Weakly Supervised Data to Improve End-to-End Speech-to-Text Translationcited by 57
  2. 2022Enhanced Direct Speech-to-Speech Translation Using Self-supervised Pre-training and Data Augmentationcited by 44
  3. 2020UWSpeech: Speech to Speech Translation for Unwritten Languagescited by 43
  4. 2020Self-Training for End-to-End Speech Translationcited by 36
  5. 2021Large-Scale Self- and Semi-Supervised Learning for Speech Translationcited by 36
  6. 2019Speech-to-speech Translation between Untranscribed Unknown Languagescited by 27
  7. 2019Analyzing ASR pretraining for low-resource speech-to-text translationcited by 27
  8. 2021Searchable Hidden Intermediates for End-to-End Models of Decomposable Sequence Taskscited by 24
  9. 2022GigaST: A 10,000-hour Pseudo Speech Translation Corpuscited by 20
  10. 2019Robust Neural Machine Translation for Clean and Noisy Speech Transcriptscited by 13
  11. 2019Self-Attentional Models for Lattice Inputscited by 12
  12. 2021Fast-MD: Fast Multi-Decoder End-to-End Speech Translation with Non-Autoregressive Hidden Intermediatescited by 9
  13. 2019Breaking the Data Barrier: Towards Robust Speech Translation via Adversarial Stability Trainingcited by 7
  14. 2021Faster Algorithms for Longest Common Substringcited by 2

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 2; 2026: 2.

4 papers.

  1. 2025Hearing to Translate: The Effectiveness of Speech Modality Integration into LLMscited by 7
  2. 2026Hearing Between the Lines: Unlocking the Reasoning Power of LLMs for Speech Evaluationcited by 5
  3. 2026LoASR-Bench: Evaluating Large Speech Language Models on Low-Resource Automatic Speech Recognition Across Language Familiescited by 1
  4. 2025It's Not a Walk in the Park! Challenges of Idiom Translation in Speech-to-text Systemscited by 0