mlsanityData to 7 Oct 2026

A multilingual training strategy for low resource Text to Speech

arXiv 2409.01217 ↗First posted 2 Sep 2024cs.CL cs.LG cs.SDcited by 3

Not on the map. Why?

Abstract (arXiv)

Recent speech technologies have led to produce high quality synthesised speech due to recent advances in neural Text to Speech (TTS). However, such TTS models depend on extensive amounts of data that can be costly to produce and is hardly scalable to all existing languages, especially that seldom attention is given to low resource languages. With techniques such as knowledge transfer, the burden of creating datasets can be alleviated. In this paper, we therefore investigate two aspects; firstly, whether data from social media can be used for a small TTS dataset construction, and secondly whether cross lingual transfer learning (TL) for a low resource language can work with this type of data. In this aspect, we specifically assess to what extent multilingual modeling can be leveraged as an alternative to training on monolingual corporas. To do so, we explore how data from foreign languages may be selected and pooled to train a TTS model for a target low resource language. Our findings show that multilingual pre-training is better than monolingual pre-training at increasing the intelligibility and naturalness of the generated speech.

Key facts. Cited by 3 papers in the map (data through 7 Oct 2026). Builds on 39 of its 55 references that are in the map. First posted on arXiv 2 Sep 2024.

Builds on

39 of 55 references.

References · 39

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2020wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representationscited by 3,853
  3. 2018VoxCeleb2: Deep Speaker Recognitioncited by 862
  4. 2017Natural TTS Synthesis by Conditioning WaveNet on Mel Spectrogram Predictionscited by 800
  5. 2020FastSpeech 2: Fast and High-Quality End-to-End Text to Speechcited by 710
  6. 2019LibriTTS: A Corpus Derived from LibriSpeech for Text-to-Speechcited by 639
  7. 2021XLS-R: Self-supervised Cross-lingual Speech Representation Learning at Scalecited by 589
  8. 2020MLS: A Large-Scale Multilingual Dataset for Speech Researchcited by 467
  9. 2015Attention-Based Models for Speech Recognitioncited by 400
  10. 2019Parallel WaveGAN: A fast waveform generation model based on generative adversarial networks with multi-resolution spectrogramcited by 337
  11. 2021Generative Spoken Language Modeling from Raw Audiocited by 336
  12. 2021DNSMOS P.835: A Non-Intrusive Perceptual Objective Speech Quality Metric to Evaluate Noise Suppressorscited by 165
27 more
  1. 2019Learning to Speak Fluently in a Foreign Language: Multilingual Speech Synthesis and Cross-Language Voice Cloningcited by 66
  2. 2017Efficiently Trainable Text-to-Speech System Based on Deep Convolutional Networks with Guided Attentioncited by 57
  3. 2019CSS10: A Collection of Single Speaker Speech Datasets for 10 Languagescited by 54
  4. 2021PnG BERT: Augmented BERT on Phonemes and Graphemes for Neural TTScited by 52
  5. 2017Listening while Speaking: Speech Chain by Deep Learningcited by 46
  6. 2020LRSpeech: Extremely Low-Resource Speech Synthesis and Recognitioncited by 43
  7. 2019Almost Unsupervised Text to Speech and Automatic Speech Recognitioncited by 41
  8. 2019The Zero Resource Speech Challenge 2019: TTS without Tcited by 38
  9. 2022BibleTTS: a large, high-fidelity, multilingual, and uniquely African speech corpuscited by 31
  10. 2018Semi-Supervised Training for Improving Data Efficiency in End-to-End Speech Synthesiscited by 30
  11. 2019End-to-end Text-to-speech for Low-resource Languages by Cross-Lingual Transfer Learningcited by 30
  12. 2019Exploring Transfer Learning for Low Resource Emotional TTScited by 21
  13. 2018Effect of data reduction on sequence-to-sequence neural TTScited by 16
  14. 2019Towards Unsupervised Speech Recognition and Synthesis with Quantized Speech Representation Learningcited by 16
  15. 2020Unsupervised Learning For Sequence-to-sequence Text-to-speech For Low-resource Languagescited by 16
  16. 2023CML-TTS A Multilingual Dataset for Speech Synthesis in Low-Resource Languagescited by 15
  17. 2018Neural Speech Synthesis with Transformer Networkcited by 14
  18. 2022Unsupervised Text-to-Speech Synthesis by Unsupervised Automatic Speech Recognitioncited by 14
  19. 2023ClArTTS: An Open-Source Classical Arabic Text-to-Speech Corpuscited by 14
  20. 2021Multilingual Byte2Speech Models for Scalable Low-resource Speech Synthesiscited by 13
  21. 2018Learning pronunciation from a foreign language in speech synthesis networkscited by 12
  22. 2022Simple and Effective Unsupervised Speech Synthesiscited by 12
  23. 2019Towards Transfer Learning for End-to-End Speech Synthesis from Deep Pre-Trained Language Modelscited by 10
  24. 2019Training Multi-Speaker Neural Text-to-Speech Systems using Speaker-Imbalanced Speech Corporacited by 8
  25. 2020Efficient neural speech synthesis for low-resource languages through multilingual modelingcited by 8
  26. 2021Cross-lingual Transfer for Speech Processing using Acoustic Language Similaritycited by 8
  27. 2023Unsupervised Data Selection for TTS: Using Arabic Broadcast News as a Case Studycited by 4

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 1; 2025: 1; 2026: 1.

3 papers.

  1. 2025Large Multimodal Models for Low-Resource Languages: A Surveycited by 4
  2. 2024ELAICHI: Enhancing Low-resource TTS by Addressing Infrequent and Low-frequency Character Bigramscited by 2
  3. 2026Deterministic Prompting for Speaker-Stable Low-Resource Greek TTScited by 0