# Extending Multilingual Speech Synthesis to 100+ Languages without Transcribed Data

arXiv: https://arxiv.org/abs/2402.18932 · HTML: https://mlsanity.com/paper/2402.18932/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 29 Feb 2024.

## Built on by (most cited first, 9 of 9)

- [FLEURS-R: A Restored Multilingual Speech Corpus for Generation Tasks](https://mlsanity.com/paper/2408.06227/) (2024)
- [Utilizing TTS Synthesized Data for Efficient Development of Keyword Spotting Model](https://mlsanity.com/paper/2407.18879/) (2024)
- [DAST: Difficulty-Aware Self-Training on Large Language Models](https://mlsanity.com/paper/2503.09029/) (2025)
- [VoiceCraft-X: Unifying Multilingual, Voice-Cloning Speech Synthesis and Speech Editing](https://mlsanity.com/paper/2511.12347/) (2025)
- [Synth4Kws: Synthesized Speech for User Defined Keyword Spotting in Low Resource Environments](https://mlsanity.com/paper/2407.16840/) (2024)
- [GE2E-KWS: Generalized End-to-End Training and Evaluation for Zero-shot Keyword Spotting](https://mlsanity.com/paper/2410.16647/) (2024)
- [Adversarial training of Keyword Spotting to Minimize TTS Data Overfitting](https://mlsanity.com/paper/2408.10463/) (2024)
- [Everyday Speech in the Indian Subcontinent](https://mlsanity.com/paper/2410.10508/) (2024)
- [Kinship in Speech: Leveraging Linguistic Relatedness for Zero-Shot TTS in Indian Languages](https://mlsanity.com/paper/2506.03884/) (2025)

## Abstract (arXiv)

Collecting high-quality studio recordings of audio is challenging, which limits the language coverage of text-to-speech (TTS) systems. This paper proposes a framework for scaling a multilingual TTS model to 100+ languages using found data without supervision. The proposed framework combines speech-text encoder pretraining with unsupervised training using untranscribed speech and unspoken text data sources, thereby leveraging massively multilingual joint speech and text representation learning. Without any transcribed speech in a new language, this TTS model can generate intelligible speech in >30 unseen languages (CER difference of <10% to ground truth). With just 15 minutes of transcribed, found data, we can reduce the intelligibility difference to 1% or less from the ground-truth, and achieve naturalness scores that match the ground-truth in several languages.