mlsanityData to 7 Oct 2026

Tackling data scarcity in speech translation using zero-shot multilingual machine translation techniques

arXiv 2201.11172 ↗First posted 26 Jan 2022cs.CL cs.SD eess.AScited by 4

Not on the map. Why?

Abstract (arXiv)

Recently, end-to-end speech translation (ST) has gained significant attention as it avoids error propagation. However, the approach suffers from data scarcity. It heavily depends on direct ST data and is less efficient in making use of speech transcription and text translation data, which is often more easily available. In the related field of multilingual text translation, several techniques have been proposed for zero-shot translation. A main idea is to increase the similarity of semantically similar sentences in different languages. We investigate whether these ideas can be applied to speech translation, by building ST models trained on speech transcription and text translation data. We investigate the effects of data augmentation and auxiliary loss function. The techniques were successfully applied to few-shot ST using limited ST data, with improvements of up to +12.9 BLEU points compared to direct end-to-end ST and +3.1 BLEU points compared to ST models fine-tuned from ASR model.

Key facts. Cited by 4 papers in the map (data through 7 Oct 2026). Builds on 14 of its 17 references that are in the map. First posted on arXiv 26 Jan 2022.

Builds on

14 of 17 references.

References · 14

  1. 2017Attention Is All You Needcited by 59,932
  2. 2018SentencePiece: A simple and language independent subword tokenizer and detokenizer for Neural Text Processingcited by 1,973
  3. 2018A Call for Clarity in Reporting BLEU Scorescited by 1,790
  4. 2016Google's Multilingual Neural Machine Translation System: Enabling Zero-Shot Translationcited by 467
  5. 2020Speech Translation and the End-to-End Promise: Taking Stock of Where We Arecited by 77
  6. 2020CoVoST 2 and Massively Multilingual Speech-to-Text Translationcited by 73
  7. 2018Pre-training on high-resource speech recognition improves low-resource speech-to-text translationcited by 70
  8. 2018Leveraging Weakly Supervised Data to Improve End-to-End Speech-to-Text Translationcited by 57
  9. 2020A General Multi-Task Learning Framework to Leverage Text Data for Speech to Text Taskscited by 51
  10. 2019Very Deep Self-Attention Networks for End-to-End Speech Recognitioncited by 46
  11. 2020Improving Zero-Shot Translation by Disentangling Positional Informationcited by 35
  12. 2019Improving Zero-shot Translation with Language-Independent Constraintscited by 33
2 more
  1. 2018Low-Resource Speech-to-Text Translationcited by 12
  2. 2019VizSeq: A Visual Analysis Toolkit for Text Generation Taskscited by 10

Built on by

Cited by year

Citing papers per year: 2022: 1; 2023: 1; 2024: 2.

4 papers.

  1. 2023CMOT: Cross-modal Mixup via Optimal Transport for Speech Translationcited by 22
  2. 2022T-Modules: Translation Modules for Zero-Shot Cross-Modal Machine Translationcited by 16
  3. 2024Pushing the Limits of Zero-shot End-to-End Speech Translationcited by 14
  4. 2024How do Multimodal Foundation Models Encode Text and Speech? An Analysis of Cross-Lingual and Cross-Modal Representationscited by 8