mlsanityData to 7 Oct 2026

Multilingual Byte2Speech Models for Scalable Low-resource Speech Synthesis

arXiv 2103.03541 ↗First posted 5 Mar 2021cs.CL cs.LG cs.SDcited by 13

Not on the map. Why?

Abstract (arXiv)

To scale neural speech synthesis to various real-world languages, we present a multilingual end-to-end framework that maps byte inputs to spectrograms, thus allowing arbitrary input scripts. Besides strong results on 40+ languages, the framework demonstrates capabilities to adapt to new languages under extreme low-resource and even few-shot scenarios of merely 40s transcribed recording, without the need of per-language resources like lexicon, extra corpus, auxiliary models, or linguistic expertise, thus ensuring scalability. While it retains satisfactory intelligibility and naturalness matching rich-resource models. Exhaustive comparative and ablation studies are performed to reveal the potential of the framework for low-resource languages. Furthermore, we propose a novel method to extract language-specific sub-networks in a multilingual model for a better understanding of its mechanism.

Key facts. Cited by 13 papers in the map (data through 7 Oct 2026). Builds on 19 of its 29 references that are in the map. First posted on arXiv 5 Mar 2021.

Builds on

19 of 29 references.

References · 19

  1. 2017Natural TTS Synthesis by Conditioning WaveNet on Mel Spectrogram Predictionscited by 800
  2. 2016Google's Multilingual Neural Machine Translation System: Enabling Zero-Shot Translationcited by 467
  3. 2019Learning to Speak Fluently in a Foreign Language: Multilingual Speech Synthesis and Cross-Language Voice Cloningcited by 66
  4. 2018Bytes are All You Need: End-to-End Multilingual Speech Recognition and Synthesis with Bytescited by 54
  5. 2019How Language-Neutral is Multilingual BERT?cited by 51
  6. 2020LRSpeech: Extremely Low-Resource Speech Synthesis and Recognitioncited by 43
  7. 2020MultiSpeech: Multi-Speaker Text to Speech with Transformercited by 42
  8. 2019Almost Unsupervised Text to Speech and Automatic Speech Recognitioncited by 41
  9. 2018Semi-Supervised Training for Improving Data Efficiency in End-to-End Speech Synthesiscited by 30
  10. 2019End-to-end Text-to-speech for Low-resource Languages by Cross-Lingual Transfer Learningcited by 30
  11. 2020One Model, Many Languages: Meta-learning for Multilingual Text-to-Speechcited by 28
  12. 2020Probing Multilingual BERT for Genetic and Typological Signalscited by 26
7 more
  1. 2020Phonological Features for 0-shot Multilingual Speech Synthesiscited by 20
  2. 2019Towards Unsupervised Speech Recognition and Synthesis with Quantized Speech Representation Learningcited by 16
  3. 2020Unsupervised Learning For Sequence-to-sequence Text-to-speech For Low-resource Languagescited by 16
  4. 2019Unsupervised Polyglot Text To Speechcited by 10
  5. 2020Efficient neural speech synthesis for low-resource languages through multilingual modelingcited by 8
  6. 2020Cross-lingual Multispeaker Text-to-Speech under Limited-Data Scenariocited by 6
  7. 2020Investigation of learning abilities on linguistic features in sequence-to-sequence text-to-speech synthesiscited by 5

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 2; 2022: 4; 2023: 3; 2024: 3; 2025: 0; 2026: 1.

13 papers.

Sort:

  1. 2023Scaling Speech Technology to 1,000+ Languagescited by 401
  2. 2022Virtuoso: Massive Multilingual Speech-Text Joint Semi-Supervised Learning for Text-To-Speechcited by 15
  3. 2023Textless Unit-to-Unit training for Many-to-Many Multilingual Speech-to-Speech Translationcited by 11
  4. 2022SQuId: Measuring Speech Naturalness in Many Languagescited by 10
  5. 2023Learning to Speak from Text: Zero-Shot Multilingual Text-to-Speech with Unsupervised Text Pretrainingcited by 10
  6. 2024Rasa: Building Expressive Speech Synthesis Systems for Indian Languages in Low-resource Settingscited by 6
  7. 2026X-Voice: Enabling Everyone to Speak 30 Languages via Zero-Shot Cross-Lingual Voice Cloningcited by 4
  8. 2021Neural Lexicon Reader: Reduce Pronunciation Errors in End-to-end TTS by Leveraging External Textual Knowledgecited by 3
  9. 2024A multilingual training strategy for low resource Text to Speechcited by 3
  10. 2021Towards Lifelong Learning of Multilingual Text-To-Speech Synthesiscited by 2
  11. 2022Few-Shot Cross-Lingual TTS Using Transferable Phoneme Embeddingcited by 1
  12. 2022Towards High-Quality Neural TTS for Low-Resource Languages by Learning Compact Speech Representationscited by 1
1 more
  1. 2024Maximizing Data Efficiency for Cross-Lingual TTS Adaptation by Self-Supervised Representation Mixing and Embedding Initializationcited by 0