mlsanityData to 7 Oct 2026

Joint Training And Decoding for Multilingual End-to-End Simultaneous Speech Translation

arXiv 2503.11080 ↗First posted 14 Mar 2025cs.CL cs.SD eess.AScited by 0

Not on the map. Why?

Abstract (arXiv)

Recent studies on end-to-end speech translation(ST) have facilitated the exploration of multilingual end-to-end ST and end-to-end simultaneous ST. In this paper, we investigate end-to-end simultaneous speech translation in a one-to-many multilingual setting which is closer to applications in real scenarios. We explore a separate decoder architecture and a unified architecture for joint synchronous training in this scenario. To further explore knowledge transfer across languages, we propose an asynchronous training strategy on the proposed unified decoder architecture. A multi-way aligned multilingual end-to-end ST dataset was curated as a benchmark testbed to evaluate our methods. Experimental results demonstrate the effectiveness of our models on the collected dataset. Our codes and data are available at: https://github.com/XiaoMi/TED-MMST.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 18 of its 25 references that are in the map. First posted on arXiv 14 Mar 2025.

Builds on

18 of 25 references.

References · 18

  1. 2017Sequence-to-Sequence Models Can Directly Translate Foreign Speechcited by 121
  2. 2016Multi-Way, Multilingual Neural Machine Translation with a Shared Attention Mechanismcited by 117
  3. 2016Listen and Translate: A Proof of Concept for End-to-End Speech-to-Text Translationcited by 113
  4. 2016Can neural machine translation do simultaneous translation?cited by 85
  5. 2020SimulMT to SimulST: Adapting Simultaneous Text Translation to End-to-End Simultaneous Speech Translationcited by 80
  6. 2020Speech Translation and the End-to-End Promise: Taking Stock of Where We Arecited by 77
  7. 2018End-to-End Automatic Speech Translation of Audiobookscited by 65
  8. 2019End-to-End Speech Translation with Knowledge Distillationcited by 58
  9. 2018Leveraging Weakly Supervised Data to Improve End-to-End Speech-to-Text Translationcited by 57
  10. 2020Curriculum Pre-training for End-to-End Speech Translationcited by 57
  11. 2020Dual-decoder Transformer for Joint Automatic Speech Recognition and Multilingual Speech Translationcited by 54
  12. 2019Multilingual End-to-End Speech Translationcited by 37
6 more
  1. 2022Large-Scale Streaming End-to-End Speech Translation with Neural Transducerscited by 31
  2. 2020Streaming Simultaneous Speech Translation with Augmented Memory Transformercited by 30
  3. 2020Adaptive Feature Selection for End-to-End Speech Translationcited by 25
  4. 2019One-To-Many Multilingual End-to-end Speech Translationcited by 17
  5. 2019Exploring Phoneme-Level Speech Representations for End-to-End Speech Translationcited by 15
  6. 2020Shallow Discourse Annotation for Chinese TED Talkscited by 13

Built on by

Not cited yet.