mlsanityData to 14 Oct 2026

Grad-StyleSpeech: Any-speaker Adaptive Text-to-Speech Synthesis with Diffusion Models

arXiv 2211.09383 ↗First posted 17 Nov 2022eess.AS cs.AI cs.SDcited by 34

On the map: Diffusion models → Audio, speech and music

Abstract (arXiv)

There has been a significant progress in Text-To-Speech (TTS) synthesis technology in recent years, thanks to the advancement in neural generative modeling. However, existing methods on any-speaker adaptive TTS have achieved unsatisfactory performance, due to their suboptimal accuracy in mimicking the target speakers' styles. In this work, we present Grad-StyleSpeech, which is an any-speaker adaptive TTS framework that is based on a diffusion model that can generate highly natural speech with extremely high similarity to target speakers' voice, given a few seconds of reference speech. Grad-StyleSpeech significantly outperforms recent speaker-adaptive TTS baselines on English benchmarks. Audio samples are available at https://nardien.github.io/grad-stylespeech-demo.

Key facts. Cited by 34 papers in the map (data through 14 Oct 2026). Placed in Audio, speech and music, one of 14 directions in Diffusion models. Builds on 22 of its 26 references that are in the map. First posted on arXiv 17 Nov 2022.

Builds on

22 of 26 references.

Closest ancestors · 4

Same direction, newest first

  1. 2021Guided-TTS: A Diffusion Model for Text-to-Speech via Classifier Guidance23 Nov
  2. 2021Diffusion-Based Voice Conversion with Fast Maximum Likelihood Sampling Scheme28 Sep
  3. 2021Grad-TTS: A Diffusion Probabilistic Model for Text-to-Speech13 May
  4. 2021Diff-TTS: A Denoising Diffusion Model for Text-to-Speech3 Apr

Other references · 18

  1. 2017Attention Is All You Needcited by 60,361
  2. 2020Denoising Diffusion Probabilistic Modelscited by 21,076
  3. 2020Score-Based Generative Modeling through Stochastic Differential Equationscited by 9,122
  4. 2020HiFi-GAN: Generative Adversarial Networks for Efficient and High Fidelity Speech Synthesiscited by 1,154
  5. 2020FastSpeech 2: Fast and High-Quality End-to-End Text to Speechcited by 717
  6. 2019LibriTTS: A Corpus Derived from LibriSpeech for Text-to-Speechcited by 649
  7. 2021Conditional Variational Autoencoder with Adversarial Learning for End-to-End Text-to-Speechcited by 561
  8. 2018Transfer Learning from Speaker Verification to Multispeaker Text-To-Speech Synthesiscited by 282
  9. 2021YourTTS: Towards Zero-Shot Multi-Speaker TTS and Zero-Shot Voice Conversion for everyonecited by 278
  10. 2020Glow-TTS: A Generative Flow for Text-to-Speech via Monotonic Alignment Searchcited by 273
  11. 2021Emotional Voice Conversion: Theory, Databases and ESDcited by 115
  12. 2018Neural Voice Cloning with a Few Samplescited by 100
6 more
  1. 2021AdaSpeech: Adaptive Text to Speech for Custom Voicecited by 98
  2. 2021Meta-StyleSpeech : Multi-Speaker Adaptive Text-to-Speech Generationcited by 93
  3. 2021One TTS Alignment To Rule Them Allcited by 47
  4. 2018Sample Efficient Adaptive Text-to-Speechcited by 45
  5. 2020MultiSpeech: Multi-Speaker Text to Speech with Transformercited by 42
  6. 2022Revisiting Over-Smoothness in Text to Speechcited by 32

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 18; 2024: 11; 2025: 4; 2026: 1.

34 papers.

Sort:

By area:

  1. 2023Neural Codec Language Models are Zero-Shot Text to Speech Synthesizerscited by 752
  2. 2023Faster Segment Anything: Towards Lightweight SAM for Mobile Applicationscited by 352
  3. 2023Matcha-TTS: A fast TTS architecture with conditional flow matchingcited by 145
  4. 2023A Survey on Audio Diffusion Models: Text To Speech Synthesis and Enhancement in Generative AIcited by 68
  5. 2024Analysis of Classifier-Free Guidance Weight Schedulerscited by 68
  6. 2023Mega-TTS: Zero-Shot Text-to-Speech at Scale with Intrinsic Inductive Biascited by 51
  7. 2023A Survey on Segment Anything Model (SAM): Vision Foundation Model Meets Prompt Engineeringcited by 45
  8. 2023MobileSAMv2: Faster Segment Anything to Everythingcited by 42
  9. 2023HierSpeech++: Bridging the Gap between Semantic and Acoustic Representation of Speech by Hierarchical Variational Inference for Zero-shot Speech Synthesiscited by 35
  10. 2023Robustness of SAM: Segment Anything Under Corruptions and Beyondcited by 31
  11. 2024Towards Controllable Speech Synthesis in the Era of Large Language Models: A Systematic Surveycited by 23
  12. 2023The Ethical Implications of Generative Audio Models: A Systematic Literature Reviewcited by 18
22 more
  1. 2023UnitSpeech: Speaker-adaptive Speech Synthesis with Untranscribed Datacited by 15
  2. 2024CoVoMix: Advancing Zero-Shot Speech Generation for Human-like Multi-talker Conversationscited by 15
  3. 2023ZET-Speech: Zero-shot adaptive Emotion-controllable Text-to-Speech Synthesis with Diffusion and Style-based Modelscited by 14
  4. 2023Diff-TTSG: Denoising probabilistic integrated speech and gesture synthesiscited by 13
  5. 2023Understanding Segment Anything Model: SAM is Biased Towards Texture Rather than Shapecited by 10
  6. 2023Image Synthesis under Limited Data: A Survey and Taxonomycited by 9
  7. 2024PosSAM: Panoptic Open-vocabulary Segment Anythingcited by 9
  8. 2025From Faces to Voices: Learning Hierarchical Representations for High-quality Video-to-Speechcited by 9
  9. 2024DurFlex-EVC: Duration-Flexible Emotional Voice Conversion Leveraging Discrete Representations without Text Alignmentcited by 8
  10. 2023Attack-SAM: Towards Attacking Segment Anything Model With Adversarial Examplescited by 5
  11. 2024A Survey of Deep Learning Audio Generation Methodscited by 5
  12. 2023SAM Meets UAP: Attacking Segment Anything Model With Universal Adversarial Perturbationcited by 4
  13. 2025Voice Cloning: Comprehensive Surveycited by 4
  14. 2024Towards Zero-Shot Text-To-Speech for Arabic Dialectscited by 3
  15. 2024Enhancing Emotional Text-to-Speech Controllability with Natural Language Guidance through Contrastive Learning and Diffusion Modelscited by 3
  16. 2024Stable-TTS: Stable Speaker-Adaptive Text-to-Speech Synthesis via Prosody Promptingcited by 3
  17. 2026EmoTra-TTS: Smooth Intra-Utterance Emotion Transitions for Speech Synthesiscited by 3
  18. 2023Face-StyleSpeech: Enhancing Zero-shot Speech Synthesis from Face Images with Improved Face-to-Speech Mappingcited by 2
  19. 2025NE-PADD: Leveraging Named Entity Knowledge for Robust Partial Audio Deepfake Detection via Attention Aggregationcited by 1
  20. 2024Improving Robustness of Diffusion-Based Zero-Shot Speech Synthesis via Stable Formant Generationcited by 0
  21. 2024CoDiff-VC: A Codec-Assisted Diffusion Model for Zero-shot Voice Conversioncited by 0
  22. 2025Towards Lightweight and Stable Zero-shot TTS with Self-distilled Representation Disentanglementcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024AS-Speech: Adaptive Style For Speech Synthesis ↗
  2. 2024StyleTTS-ZS: Efficient High-Quality Zero-Shot Text-to-Speech Synthesis with Distilled Time-Varying Style Diffusion
  3. 2023StyleTTS 2: Towards Human-Level Text-to-Speech through Style Diffusion and Adversarial Training with Large Speech Language Models
  4. 2022NoreSpeech: Knowledge Distillation based Conditional Diffusion Model for Noise-robust Expressive TTS
  5. 2023Multi-GradSpeech: Towards Diffusion-based Multi-Speaker Text-to-speech Using Consistent Diffusion Models
  6. 2023Stylebook: Content-Dependent Speaking Style Modeling for Any-to-Any Voice Conversion using Only Speech Data
  7. 2021AdaSpeech 3: Adaptive Text to Speech for Spontaneous Style
  8. 2022Guided-TTS 2: A Diffusion Model for High-quality Adaptive Text-to-Speech with Untranscribed Data