mlsanityData to 7 Oct 2026

Zero-Shot vs. Few-Shot Multi-Speaker TTS Using Pre-trained Czech SpeechT5 Model

arXiv 2407.17167 ↗First posted 24 Jul 2024cs.SD cs.CL eess.AScited by 0

Not on the map. Why?

Abstract (arXiv)

In this paper, we experimented with the SpeechT5 model pre-trained on large-scale datasets. We pre-trained the foundation model from scratch and fine-tuned it on a large-scale robust multi-speaker text-to-speech (TTS) task. We tested the model capabilities in a zero- and few-shot scenario. Based on two listening tests, we evaluated the synthetic audio quality and the similarity of how synthetic voices resemble real voices. Our results showed that the SpeechT5 model can generate a synthetic voice for any speaker using only one minute of the target speaker's data. We successfully demonstrated the high quality and similarity of our synthetic voices on publicly known Czech politicians and celebrities.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 18 of its 30 references that are in the map. First posted on arXiv 24 Jul 2024.

Builds on

18 of 30 references.

References · 18

  1. 2017Attention Is All You Needcited by 59,932
  2. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
  3. 2020wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representationscited by 3,853
  4. 2019Common Voice: A Massively-Multilingual Speech Corpuscited by 1,358
  5. 2023Neural Codec Language Models are Zero-Shot Text to Speech Synthesizerscited by 737
  6. 2019LibriTTS: A Corpus Derived from LibriSpeech for Text-to-Speechcited by 639
  7. 2021XLS-R: Self-supervised Cross-lingual Speech Representation Learning at Scalecited by 589
  8. 2021VoxPopuli: A Large-Scale Multilingual Speech Corpus for Representation Learning, Semi-Supervised Learning and Interpretationcited by 455
  9. 2023Voicebox: Text-Guided Multilingual Universal Speech Generation at Scalecited by 373
  10. 2021YourTTS: Towards Zero-Shot Multi-Speaker TTS and Zero-Shot Voice Conversion for everyonecited by 277
  11. 2023NaturalSpeech 2: Latent Diffusion Models are Natural and Zero-Shot Speech and Singing Synthesizerscited by 233
  12. 2024NaturalSpeech 3: Zero-Shot Speech Synthesis with Factorized Codec and Diffusion Modelscited by 222
6 more
  1. 2023StyleTTS 2: Towards Human-Level Text-to-Speech through Style Diffusion and Adversarial Training with Large Speech Language Modelscited by 148
  2. 2023UniAudio: An Audio Foundation Model Toward Universal Audio Generationcited by 127
  3. 2023Mega-TTS 2: Boosting Prompting Mechanisms for Zero-Shot Speech Synthesiscited by 41
  4. 2023HierSpeech++: Bridging the Gap between Semantic and Acoustic Representation of Speech by Hierarchical Variational Inference for Zero-shot Speech Synthesiscited by 35
  5. 2023The VoiceMOS Challenge 2023: Zero-shot Subjective Speech Quality Prediction for Multiple Domainscited by 8
  6. 2022Exploring Capabilities of Monolingual Audio Transformers using Large Datasets in Automatic Speech Recognition of Czechcited by 3

Built on by

Not cited yet.