# End-to-End Speech Translation for Low-Resource Languages Using Weakly Labeled Data

arXiv: https://arxiv.org/abs/2506.16251 · HTML: https://mlsanity.com/paper/2506.16251/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 17 of its 28 references that are in the map.
- First posted on arXiv 19 Jun 2025.

## Builds on (most cited first, 17 of 17)

- [SentencePiece: A simple and language independent subword tokenizer and detokenizer for Neural Text Processing](https://mlsanity.com/paper/1808.06226/) (2018)
- [A Call for Clarity in Reporting BLEU Scores](https://mlsanity.com/paper/1804.08771/) (2018)
- [Common Voice: A Massively-Multilingual Speech Corpus](https://mlsanity.com/paper/1912.06670/) (2019)
- [No Language Left Behind: Scaling Human-Centered Machine Translation](https://mlsanity.com/paper/2207.04672/) (2022)
- [W2v-BERT: Combining Contrastive Learning and Masked Language Modeling for Self-Supervised Speech Pre-Training](https://mlsanity.com/paper/2108.06209/) (2021)
- [CCMatrix: Mining Billions of High-Quality Parallel Sentences on the WEB](https://mlsanity.com/paper/1911.04944/) (2019)
- [SONAR: Sentence-Level Multimodal and Language-Agnostic Representations](https://mlsanity.com/paper/2308.11466/) (2023)
- [ESPnet-ST: All-in-One Speech Translation Toolkit](https://mlsanity.com/paper/2004.10234/) (2020)
- [UnitY: Two-pass Direct Speech-to-speech Translation with Discrete Units](https://mlsanity.com/paper/2212.08055/) (2022)
- [SpeechMatrix: A Large-Scale Mined Corpus of Multilingual Speech-to-Speech Translations](https://mlsanity.com/paper/2211.04508/) (2022)
- [Analyzing ASR pretraining for low-resource speech-to-text translation](https://mlsanity.com/paper/1910.10762/) (2019)
- [Low-Resource Corpus Filtering using Multilingual Sentence Embeddings](https://mlsanity.com/paper/1906.08885/) (2019)
- [Leveraging unsupervised and weakly-supervised data to improve direct speech-to-speech translation](https://mlsanity.com/paper/2203.13339/) (2022)
- [Findings of the IWSLT 2024 Evaluation Campaign](https://mlsanity.com/paper/2411.05088/) (2024)
- [Effectiveness of Mining Audio and Text Pairs from Public Data for Improving ASR Systems for Low-Resource Languages](https://mlsanity.com/paper/2208.12666/) (2022)
- [Speech Resources in the Tamasheq Language](https://mlsanity.com/paper/2201.05051/) (2022)
- [Strategies for improving low resource speech to text translation relying on pre-trained ASR models](https://mlsanity.com/paper/2306.00208/) (2023)

## Built on by (most cited first, 1 of 1)

- [FLiP: Towards understanding and interpreting multimodal multilingual sentence embeddings](https://mlsanity.com/paper/2604.18109/) (2026)

## Abstract (arXiv)

The scarcity of high-quality annotated data presents a significant challenge in developing effective end-to-end speech-to-text translation (ST) systems, particularly for low-resource languages. This paper explores the hypothesis that weakly labeled data can be used to build ST models for low-resource language pairs. We constructed speech-to-text translation datasets with the help of bitext mining using state-of-the-art sentence encoders. We mined the multilingual Shrutilipi corpus to build Shrutilipi-anuvaad, a dataset comprising ST data for language pairs Bengali-Hindi, Malayalam-Hindi, Odia-Hindi, and Telugu-Hindi. We created multiple versions of training data with varying degrees of quality and quantity to investigate the effect of quality versus quantity of weakly labeled data on ST model performance. Results demonstrate that ST systems can be built using weakly labeled data, with performance comparable to massive multi-modal multilingual baselines such as SONAR and SeamlessM4T.