mlsanityData to 14 Oct 2026

End-to-End Speech Translation for Low-Resource Languages Using Weakly Labeled Data

arXiv 2506.16251 ↗First posted 19 Jun 2025cs.CL eess.AScited by 1

Not on the map. Why?

Abstract (arXiv)

The scarcity of high-quality annotated data presents a significant challenge in developing effective end-to-end speech-to-text translation (ST) systems, particularly for low-resource languages. This paper explores the hypothesis that weakly labeled data can be used to build ST models for low-resource language pairs. We constructed speech-to-text translation datasets with the help of bitext mining using state-of-the-art sentence encoders. We mined the multilingual Shrutilipi corpus to build Shrutilipi-anuvaad, a dataset comprising ST data for language pairs Bengali-Hindi, Malayalam-Hindi, Odia-Hindi, and Telugu-Hindi. We created multiple versions of training data with varying degrees of quality and quantity to investigate the effect of quality versus quantity of weakly labeled data on ST model performance. Results demonstrate that ST systems can be built using weakly labeled data, with performance comparable to massive multi-modal multilingual baselines such as SONAR and SeamlessM4T.

Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Builds on 17 of its 28 references that are in the map. First posted on arXiv 19 Jun 2025.

Builds on

17 of 28 references.

References · 17

  1. 2018SentencePiece: A simple and language independent subword tokenizer and detokenizer for Neural Text Processingcited by 1,989
  2. 2018A Call for Clarity in Reporting BLEU Scorescited by 1,799
  3. 2019Common Voice: A Massively-Multilingual Speech Corpuscited by 1,384
  4. 2022No Language Left Behind: Scaling Human-Centered Machine Translationcited by 1,312
  5. 2021W2v-BERT: Combining Contrastive Learning and Masked Language Modeling for Self-Supervised Speech Pre-Trainingcited by 347
  6. 2019CCMatrix: Mining Billions of High-Quality Parallel Sentences on the WEBcited by 177
  7. 2023SONAR: Sentence-Level Multimodal and Language-Agnostic Representationscited by 108
  8. 2020ESPnet-ST: All-in-One Speech Translation Toolkitcited by 82
  9. 2022UnitY: Two-pass Direct Speech-to-speech Translation with Discrete Unitscited by 60
  10. 2022SpeechMatrix: A Large-Scale Mined Corpus of Multilingual Speech-to-Speech Translationscited by 35
  11. 2019Analyzing ASR pretraining for low-resource speech-to-text translationcited by 27
  12. 2019Low-Resource Corpus Filtering using Multilingual Sentence Embeddingscited by 25
5 more
  1. 2022Leveraging unsupervised and weakly-supervised data to improve direct speech-to-speech translationcited by 20
  2. 2024Findings of the IWSLT 2024 Evaluation Campaigncited by 19
  3. 2022Effectiveness of Mining Audio and Text Pairs from Public Data for Improving ASR Systems for Low-Resource Languagescited by 16
  4. 2022Speech Resources in the Tamasheq Languagecited by 13
  5. 2023Strategies for improving low resource speech to text translation relying on pre-trained ASR modelscited by 3

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 1.

1 paper.

  1. 2026FLiP: Towards understanding and interpreting multimodal multilingual sentence embeddingscited by 0