mlsanityData to 7 Oct 2026

MParrotTTS: Multilingual Multi-speaker Text to Speech Synthesis in Low Resource Setting

arXiv 2305.11926 ↗First posted 19 May 2023cs.SD cs.CL cs.LGcited by 0

Not on the map. Why?

Abstract (arXiv)

We present MParrotTTS, a unified multilingual, multi-speaker text-to-speech (TTS) synthesis model that can produce high-quality speech. Benefiting from a modularized training paradigm exploiting self-supervised speech representations, MParrotTTS adapts to a new language with minimal supervised data and generalizes to languages not seen while training the self-supervised backbone. Moreover, without training on any bilingual or parallel examples, MParrotTTS can transfer voices across languages while preserving the speaker-specific characteristics, e.g., synthesizing fluent Hindi speech using a French speaker's voice and accent. We present extensive results on six languages in terms of speech naturalness and speaker similarity in parallel and cross-lingual synthesis. The proposed model outperforms the state-of-the-art multilingual TTS models and baselines, using only a small fraction of supervised training data. Speech samples from our model can be found at https://paper2438.github.io/tts/

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 19 of its 26 references that are in the map. First posted on arXiv 19 May 2023.

Builds on

19 of 26 references.

References · 19

  1. 2017Attention Is All You Needcited by 59,932
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  3. 2015Domain-Adversarial Training of Neural Networkscited by 3,023
  4. 2021HuBERT: Self-Supervised Speech Representation Learning by Masked Prediction of Hidden Unitscited by 2,301
  5. 2020HiFi-GAN: Generative Adversarial Networks for Efficient and High Fidelity Speech Synthesiscited by 1,139
  6. 2020FastSpeech 2: Fast and High-Quality End-to-End Text to Speechcited by 710
  7. 2017Tacotron: Towards End-to-End Speech Synthesiscited by 487
  8. 2021VoxPopuli: A Large-Scale Multilingual Speech Corpus for Representation Learning, Semi-Supervised Learning and Interpretationcited by 455
  9. 2021Speech Resynthesis from Discrete Disentangled Self-Supervised Representationscited by 243
  10. 2021Textless Speech-to-Speech Translation on Real Datacited by 121
  11. 2022CVSS Corpus and Massively Multilingual Speech-to-Speech Translationcited by 71
  12. 2019Learning to Speak Fluently in a Foreign Language: Multilingual Speech Synthesis and Cross-Language Voice Cloningcited by 66
7 more
  1. 2020LRSpeech: Extremely Low-Resource Speech Synthesis and Recognitioncited by 43
  2. 2021Investigating on Incorporating Pretrained and Learnable Speaker Representations for Multi-Speaker Multi-Style Text-to-Speechcited by 37
  3. 2021Parallel Tacotron 2: A Non-Autoregressive Neural TTS Model with Differentiable Duration Modelingcited by 31
  4. 2020One Model, Many Languages: Meta-learning for Multilingual Text-to-Speechcited by 28
  5. 2020Unsupervised Learning For Sequence-to-sequence Text-to-speech For Low-resource Languagescited by 16
  6. 2022Heterogeneous Target Speech Separationcited by 13
  7. 2019Cross-lingual Multi-speaker Text-to-speech Synthesis for Voice Cloning without Using Parallel Corpus for Unseen Speakerscited by 11

Built on by

Not cited yet.