mlsanityData to 7 Oct 2026

YourTTS: Towards Zero-Shot Multi-Speaker TTS and Zero-Shot Voice Conversion for everyone

arXiv 2112.02418 ↗First posted 4 Dec 2021cs.SD cs.CL eess.AScited by 277

Not on the map. Why?

Cited by 35 in Diffusion models, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

YourTTS brings the power of a multilingual approach to the task of zero-shot multi-speaker TTS. Our method builds upon the VITS model and adds several novel modifications for zero-shot multi-speaker and multilingual training. We achieved state-of-the-art (SOTA) results in zero-shot multi-speaker TTS and results comparable to SOTA in zero-shot voice conversion on the VCTK dataset. Additionally, our approach achieves promising results in a target language with a single-speaker dataset, opening possibilities for zero-shot multi-speaker TTS and zero-shot voice conversion systems in low-resource languages. Finally, it is possible to fine-tune the YourTTS model with less than 1 minute of speech and achieve state-of-the-art results in voice similarity and with reasonable quality. This is important to allow synthesis for speakers with a very different voice or recording characteristics from those seen during training.

Key facts. Cited by 277 papers in the map (data through 7 Oct 2026). Builds on 34 of its 49 references that are in the map. First posted on arXiv 4 Dec 2021.

Builds on

34 of 49 references.

References · 34

  1. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,160
  2. 2020wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representationscited by 3,853
  3. 2016Density estimation using Real NVPcited by 2,029
  4. 2016WaveNet: A Generative Model for Raw Audiocited by 1,710
  5. 2019Common Voice: A Massively-Multilingual Speech Corpuscited by 1,358
  6. 2020HiFi-GAN: Generative Adversarial Networks for Efficient and High Fidelity Speech Synthesiscited by 1,139
  7. 2018VoxCeleb2: Deep Speaker Recognitioncited by 862
  8. 2017Natural TTS Synthesis by Conditioning WaveNet on Mel Spectrogram Predictionscited by 800
  9. 2017VoxCeleb: a large-scale speaker identification datasetcited by 714
  10. 2020FastSpeech 2: Fast and High-Quality End-to-End Text to Speechcited by 710
  11. 2019LibriTTS: A Corpus Derived from LibriSpeech for Text-to-Speechcited by 639
  12. 2021Conditional Variational Autoencoder with Adversarial Learning for End-to-End Text-to-Speechcited by 555
22 more
  1. 2020MLS: A Large-Scale Multilingual Dataset for Speech Researchcited by 467
  2. 2018WaveGlow: A Flow-based Generative Network for Speech Synthesiscited by 308
  3. 2017Generalized End-to-End Loss for Speaker Verificationcited by 290
  4. 2018Transfer Learning from Speaker Verification to Multispeaker Text-To-Speech Synthesiscited by 280
  5. 2020Glow-TTS: A Generative Flow for Text-to-Speech via Monotonic Alignment Searchcited by 269
  6. 2019AUTOVC: Zero-Shot Voice Style Transfer with Only Autoencoder Losscited by 189
  7. 2021A Survey on Neural Speech Synthesiscited by 182
  8. 2020In defence of metric learning for speaker recognitioncited by 123
  9. 2018Neural Voice Cloning with a Few Samplescited by 100
  10. 2019High Fidelity Speech Synthesis with Adversarial Networkscited by 90
  11. 2021Neural Analysis and Synthesis: Reconstructing Speech from Self-Supervised Representationscited by 81
  12. 2019Zero-Shot Multi-Speaker Text-To-Speech with State-of-the-art Neural Speaker Embeddingscited by 73
  13. 2017Deep Voice 3: Scaling Text-to-Speech with Convolutional Sequence Learningcited by 67
  14. 2019Learning to Speak Fluently in a Foreign Language: Multilingual Speech Synthesis and Cross-Language Voice Cloningcited by 66
  15. 2020FullSubNet: A Full-Band and Sub-Band Fusion Model for Real-Time Single-Channel Speech Enhancementcited by 66
  16. 2021SC-GlowTTS: an Efficient Zero-Shot Multi-Speaker Text-To-Speech Modelcited by 56
  17. 2018Exploring the Encoding Layer and Loss Function in End-to-End Speaker and Language Recognition Systemcited by 45
  18. 2020Clova Baseline System for the VoxCeleb Speaker Recognition Challenge 2020cited by 36
  19. 2020One Model, Many Languages: Meta-learning for Multilingual Text-to-Speechcited by 28
  20. 2020Attentron: Few-Shot Text-to-Speech Utilizing Attention-Based Variable-Length Embeddingcited by 27
  21. 2020TTS-Portuguese Corpus: a corpus for speech synthesis in Brazilian Portuguesecited by 9
  22. 2021NoiseVC: Towards High Quality Zero-Shot Voice Conversioncited by 4

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 22; 2023: 51; 2024: 95; 2025: 69; 2026: 40.

277 papers.

Sort:

By area:

  1. 2023Neural Codec Language Models are Zero-Shot Text to Speech Synthesizerscited by 737
  2. 2022AudioLM: a Language Modeling Approach to Audio Generationcited by 611
  3. 2023Scaling Speech Technology to 1,000+ Languagescited by 401
  4. 2023Voicebox: Text-Guided Multilingual Universal Speech Generation at Scalecited by 373
  5. 2023NaturalSpeech 2: Latent Diffusion Models are Natural and Zero-Shot Speech and Singing Synthesizerscited by 233
  6. 2024NaturalSpeech 3: Zero-Shot Speech Synthesis with Factorized Codec and Diffusion Modelscited by 222
  7. 2024XTTS: a Massively Multilingual Zero-Shot Text-to-Speech Modelcited by 174
  8. 2023Speak Foreign Languages with Your Own Voice: Cross-Lingual Neural Codec Language Modelingcited by 166
  9. 2023StyleTTS 2: Towards Human-Level Text-to-Speech through Style Diffusion and Adversarial Training with Large Speech Language Modelscited by 148
  10. 2024VALL-E 2: Neural Codec Language Models are Human Parity Zero-Shot Text to Speech Synthesizerscited by 119
  11. 2024VoiceCraft: Zero-Shot Speech Editing and Text-to-Speech in the Wildcited by 118
  12. 2024ASVspoof 5: Crowdsourced Speech Data, Deepfakes, and Adversarial Attacks at Scalecited by 88
38 more
  1. 2023SpeechTokenizer: Unified Speech Tokenizer for Speech Large Language Modelscited by 85
  2. 2025IndexTTS2: A Breakthrough in Emotionally Expressive and Duration-Controlled Auto-Regressive Zero-Shot Text-to-Speechcited by 85
  3. 2022FreeVC: Towards High-Quality Text-Free One-Shot Voice Conversioncited by 79
  4. 2023AV-Deepfake1M: A Large-Scale LLM-Driven Audio-Visual Deepfake Datasetcited by 79
  5. 2024BASE TTS: Lessons from building a billion-parameter Text-to-Speech model on 100K hours of datacited by 74
  6. 2024Detecting Multimedia Generated by Large AI Models: A Surveycited by 73
  7. 2023Voice Conversion With Just Nearest Neighborscited by 67
  8. 2024Proactive Detection of Voice Cloning with Localized Watermarkingcited by 67
  9. 2024Autoregressive Speech Synthesis without Vector Quantizationcited by 65
  10. 2023SpeechX: Neural Codec Language Model as a Versatile Speech Transformercited by 64
  11. 2023Generative Pre-training for Speech with Flow Matchingcited by 55
  12. 2023Mega-TTS: Zero-Shot Text-to-Speech at Scale with Intrinsic Inductive Biascited by 51
  13. 2023PromptTTS 2: Describing and Generating Voices with Text Promptcited by 47
  14. 2025Recent Advances in Discrete Speech Tokens: A Reviewcited by 42
  15. 2022AdaSpeech 4: Adaptive Text to Speech in Zero-Shot Scenarioscited by 41
  16. 2024Let's Go Real Talk: Spoken Dialogue Model for Face-to-Face Conversationcited by 40
  17. 2024Zero-shot Voice Conversion with Diffusion Transformerscited by 39
  18. 2025MegaTTS 3: Sparse Alignment Enhanced Latent Diffusion Transformer for Zero-Shot Speech Synthesiscited by 39
  19. 2022Guided-TTS 2: A Diffusion Model for High-quality Adaptive Text-to-Speech with Untranscribed Datacited by 35
  20. 2023HierSpeech++: Bridging the Gap between Semantic and Acoustic Representation of Speech by Hierarchical Variational Inference for Zero-shot Speech Synthesiscited by 35
  21. 2024MovieBench: A Hierarchical Movie Level Dataset for Long Video Generationcited by 35
  22. 2022Grad-StyleSpeech: Any-speaker Adaptive Text-to-Speech Synthesis with Diffusion Modelscited by 34
  23. 2022StyleTTS: A Style-Based Generative Model for Natural and Diverse Text-to-Speech Synthesiscited by 32
  24. 2025SALM-Duplex: Efficient and Direct Duplex Modeling for Speech-to-Speech Language Modelcited by 32
  25. 2024DiTTo-TTS: Diffusion Transformers for Scalable Text-to-Speech without Domain-Specific Factorscited by 31
  26. 2023OpenVoice: Versatile Instant Voice Cloningcited by 30
  27. 2025ZipVoice: Fast and High-Quality Zero-Shot Text-to-Speech with Flow Matchingcited by 30
  28. 2022Towards Building Text-To-Speech Systems for the Next Billion Userscited by 27
  29. 2023Make-A-Voice: Unified Voice Synthesis With Discrete Representationcited by 27
  30. 2025IndexTTS: An Industrial-Level Controllable and Efficient Zero-Shot Text-To-Speech Systemcited by 27
  31. 2025Koel-TTS: Enhancing LLM based Speech Generation with Preference Alignment and Classifier Free Guidancecited by 26
  32. 2025Human Motion Video Generation: A Surveycited by 25
  33. 2024Circumventing shortcuts in audio-visual deepfake detection datasets with unsupervised learningcited by 23
  34. 2022Memories are One-to-Many Mapping Alleviators in Talking Face Generationcited by 22
  35. 2024ESPnet-SPK: full pipeline speaker embedding toolkit with reproducible recipes, self-supervised front-ends, and off-the-shelf modelscited by 22
  36. 2024FlashSpeech: Efficient Zero-Shot Speech Synthesiscited by 22
  37. 2024Towards Controllable Speech Synthesis in the Era of Large Language Models: A Systematic Surveycited by 22
  38. 2026OmniVoice: Towards Omnilingual Zero-Shot Text-to-Speech with Diffusion Language Modelscited by 22

50 most cited of 277 All 277 (JSON)