# FST: the FAIR Speech Translation System for the IWSLT21 Multilingual Shared Task

arXiv: https://arxiv.org/abs/2107.06959 · HTML: https://mlsanity.com/paper/2107.06959/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Builds on 18 of its 21 references that are in the map.
- First posted on arXiv 14 Jul 2021.

## Builds on (most cited first, 18 of 18)

- [Unsupervised Cross-lingual Representation Learning at Scale](https://mlsanity.com/paper/1911.02116/) (2019)
- [wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representations](https://mlsanity.com/paper/2006.11477/) (2020)
- [SentencePiece: A simple and language independent subword tokenizer and detokenizer for Neural Text Processing](https://mlsanity.com/paper/1808.06226/) (2018)
- [Common Voice: A Massively-Multilingual Speech Corpus](https://mlsanity.com/paper/1912.06670/) (2019)
- [Multilingual Denoising Pre-training for Neural Machine Translation](https://mlsanity.com/paper/2001.08210/) (2020)
- [Beyond English-Centric Multilingual Machine Translation](https://mlsanity.com/paper/2010.11125/) (2020)
- [CCNet: Extracting High Quality Monolingual Datasets from Web Crawl Data](https://mlsanity.com/paper/1911.00359/) (2019)
- [Unsupervised Cross-lingual Representation Learning for Speech Recognition](https://mlsanity.com/paper/2006.13979/) (2020)
- [MLS: A Large-Scale Multilingual Dataset for Speech Research](https://mlsanity.com/paper/2012.03411/) (2020)
- [VoxPopuli: A Large-Scale Multilingual Speech Corpus for Representation Learning, Semi-Supervised Learning and Interpretation](https://mlsanity.com/paper/2101.00390/) (2021)
- [Europarl-ST: A Multilingual Corpus For Speech Translation Of Parliamentary Debates](https://mlsanity.com/paper/1911.03167/) (2019)
- [The Multilingual TEDx Corpus for Speech Recognition and Translation](https://mlsanity.com/paper/2102.01757/) (2021)
- [Margin-based Parallel Corpus Mining with Multilingual Sentence Embeddings](https://mlsanity.com/paper/1811.01136/) (2018)
- [CoVoST: A Diverse Multilingual Speech-To-Text Translation Corpus](https://mlsanity.com/paper/2002.01320/) (2020)
- [Improving Speech Translation by Understanding and Learning from the Auxiliary Text Translation Task](https://mlsanity.com/paper/2107.05782/) (2021)
- [Learning Joint Multilingual Sentence Representations with Neural Machine Translation](https://mlsanity.com/paper/1704.04154/) (2017)
- [A General Multi-Task Learning Framework to Leverage Text Data for Speech to Text Tasks](https://mlsanity.com/paper/2010.11338/) (2020)
- [Multilingual End-to-End Speech Translation](https://mlsanity.com/paper/1910.00254/) (2019)

## Built on by (most cited first, 6 of 6)

- [LAMASSU: Streaming Language-Agnostic Multilingual Speech Recognition and Translation Using Neural Transducers](https://mlsanity.com/paper/2211.02809/) (2022)
- [ComSL: A Composite Speech-Language Model for End-to-End Speech-to-Text Translation](https://mlsanity.com/paper/2305.14838/) (2023)
- [NAVER LABS Europe's Multilingual Speech Translation Systems for the IWSLT 2023 Low-Resource Track](https://mlsanity.com/paper/2306.07763/) (2023)
- [Don't Discard Fixed-Window Audio Segmentation in Speech-to-Text Translation](https://mlsanity.com/paper/2210.13363/) (2022)
- [Multilingual Speech-to-Speech Translation into Multiple Target Languages](https://mlsanity.com/paper/2307.08655/) (2023)
- [The YiTrans End-to-End Speech Translation System for IWSLT 2022 Offline Shared Task](https://mlsanity.com/paper/2206.05777/) (2022)

## Abstract (arXiv)

In this paper, we describe our end-to-end multilingual speech translation system submitted to the IWSLT 2021 evaluation campaign on the Multilingual Speech Translation shared task. Our system is built by leveraging transfer learning across modalities, tasks and languages. First, we leverage general-purpose multilingual modules pretrained with large amounts of unlabelled and labelled data. We further enable knowledge transfer from the text task to the speech task by training two tasks jointly. Finally, our multilingual model is finetuned on speech translation task-specific data to achieve the best translation results. Experimental results show our system outperforms the reported systems, including both end-to-end and cascaded based approaches, by a large margin. In some translation directions, our speech translation results evaluated on the public Multilingual TEDx test set are even comparable with the ones from a strong text-to-text translation system, which uses the oracle speech transcripts as input.