# Multilingual End-to-End Speech Translation

arXiv: https://arxiv.org/abs/1910.00254 · HTML: https://mlsanity.com/paper/1910.00254/

## Key facts

- Cited by 37 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 1 Oct 2019.

## Built on by (most cited first, 30 of 37)

- [Recent Advances in End-to-End Automatic Speech Recognition](https://mlsanity.com/paper/2111.01690/) (2021)
- [The Multilingual TEDx Corpus for Speech Recognition and Translation](https://mlsanity.com/paper/2102.01757/) (2021)
- [Learning Shared Semantic Space for Speech-to-Text Translation](https://mlsanity.com/paper/2105.03095/) (2021)
- [Large-Scale Self- and Semi-Supervised Learning for Speech Translation](https://mlsanity.com/paper/2104.06678/) (2021)
- [SLTUNET: A Simple Unified Model for Sign Language Translation](https://mlsanity.com/paper/2305.01778/) (2023)
- [Source and Target Bidirectional Knowledge Distillation for End-to-end Speech Translation](https://mlsanity.com/paper/2104.06457/) (2021)
- [Revisiting End-to-End Speech-to-Text Translation From Scratch](https://mlsanity.com/paper/2206.04571/) (2022)
- [Pre-training for Speech Translation: CTC Meets Optimal Transport](https://mlsanity.com/paper/2301.11716/) (2023)
- [Recent Advances in Direct Speech-to-text Translation](https://mlsanity.com/paper/2306.11646/) (2023)
- [AV2AV: Direct Audio-Visual Speech to Audio-Visual Speech Translation with Unified Audio-Visual Speech Representation](https://mlsanity.com/paper/2312.02512/) (2023)
- [A Weakly-Supervised Streaming Multilingual Speech Model with Truly Zero-Shot Capability](https://mlsanity.com/paper/2211.02499/) (2022)
- [Blockwise Streaming Transformer for Spoken Language Understanding and Simultaneous Speech Translation](https://mlsanity.com/paper/2204.08920/) (2022)
- [Cross-Lingual Transfer Learning for Speech Translation](https://mlsanity.com/paper/2407.01130/) (2024)
- [LAMASSU: Streaming Language-Agnostic Multilingual Speech Recognition and Translation Using Neural Transducers](https://mlsanity.com/paper/2211.02809/) (2022)
- [End-to-End Speech-to-Text Translation: A Survey](https://mlsanity.com/paper/2312.01053/) (2023)
- [Investigating Decoder-only Large Language Models for Speech-to-text Translation](https://mlsanity.com/paper/2407.03169/) (2024)
- [How do Multimodal Foundation Models Encode Text and Speech? An Analysis of Cross-Lingual and Cross-Modal Representations](https://mlsanity.com/paper/2411.17666/) (2024)
- [Efficient Training for Multilingual Visual Speech Recognition: Pre-training with Discretized Visual Speech Representation](https://mlsanity.com/paper/2401.09802/) (2024)
- [Direct Speech Translation for Automatic Subtitling](https://mlsanity.com/paper/2209.13192/) (2022)
- [FST: the FAIR Speech Translation System for the IWSLT21 Multilingual Shared Task](https://mlsanity.com/paper/2107.06959/) (2021)
- [Discrete Cross-Modal Alignment Enables Zero-Shot Speech Translation](https://mlsanity.com/paper/2210.09556/) (2022)
- [Pay Better Attention to Attention: Head Selection in Multilingual and Multi-Domain Sequence Modeling](https://mlsanity.com/paper/2106.10840/) (2021)
- [Dodging the Data Bottleneck: Automatic Subtitling with Automatically Segmented ST Corpora](https://mlsanity.com/paper/2209.10608/) (2022)
- [Does Joint Training Really Help Cascaded Speech Translation?](https://mlsanity.com/paper/2210.13700/) (2022)
- [Towards a Deep Understanding of Multilingual End-to-End Speech Translation](https://mlsanity.com/paper/2310.20456/) (2023)
- [Multilingual Simultaneous Speech Translation](https://mlsanity.com/paper/2203.14835/) (2022)
- [How To Build Competitive Multi-gender Speech Translation Models For Controlling Speaker Gender Translation](https://mlsanity.com/paper/2310.15114/) (2023)
- [End-to-End Single-Channel Speaker-Turn Aware Conversational Speech Translation](https://mlsanity.com/paper/2311.00697/) (2023)
- [Zero-Shot End-To-End Spoken Question Answering In Medical Domain](https://mlsanity.com/paper/2406.05876/) (2024)
- [Efficient CTC Regularization via Coarse Labels for End-to-End Speech Translation](https://mlsanity.com/paper/2302.10871/) (2023)

## Abstract (arXiv)

In this paper, we propose a simple yet effective framework for multilingual end-to-end speech translation (ST), in which speech utterances in source languages are directly translated to the desired target languages with a universal sequence-to-sequence architecture. While multilingual models have shown to be useful for automatic speech recognition (ASR) and machine translation (MT), this is the first time they are applied to the end-to-end ST problem. We show the effectiveness of multilingual end-to-end ST in two scenarios: one-to-many and many-to-many translations with publicly available data. We experimentally confirm that multilingual end-to-end ST models significantly outperform bilingual ones in both scenarios. The generalization of multilingual training is also evaluated in a transfer learning scenario to a very low-resource language pair. All of our codes and the database are publicly available to encourage further research in this emergent multilingual ST topic.