mlsanityData to 14 Oct 2026

Guided-TTS 2: A Diffusion Model for High-quality Adaptive Text-to-Speech with Untranscribed Data

arXiv 2205.15370 ↗First posted 30 May 2022cs.SD cs.AI eess.AScited by 35

On the map: Diffusion models → Audio, speech and music

Abstract (arXiv)

We propose Guided-TTS 2, a diffusion-based generative model for high-quality adaptive TTS using untranscribed data. Guided-TTS 2 combines a speaker-conditional diffusion model with a speaker-dependent phoneme classifier for adaptive text-to-speech. We train the speaker-conditional diffusion model on large-scale untranscribed datasets for a classifier-free guidance method and further fine-tune the diffusion model on the reference speech of the target speaker for adaptation, which only takes 40 seconds. We demonstrate that Guided-TTS 2 shows comparable performance to high-quality single-speaker TTS baselines in terms of speech quality and speaker similarity with only a ten-second untranscribed data. We further show that Guided-TTS 2 outperforms adaptive TTS baselines on multi-speaker datasets even with a zero-shot adaptation setting. Guided-TTS 2 can adapt to a wide range of voices only using untranscribed speech, which enables adaptive TTS with the voice of non-human characters such as Gollum in textit"The Lord of the Rings".

Key facts. Cited by 35 papers in the map (data through 14 Oct 2026). Placed in Audio, speech and music, one of 14 directions in Diffusion models. Builds on 40 of its 48 references that are in the map. First posted on arXiv 30 May 2022.

Builds on

40 of 48 references.

Closest ancestors · 4

Same direction, newest first

  1. 2021PriorGrad: Improving Conditional Denoising Diffusion Models with Data-Dependent Adaptive Prior11 Jun
  2. 2021Grad-TTS: A Diffusion Probabilistic Model for Text-to-Speech13 May
  3. 2021Diff-TTS: A Denoising Diffusion Model for Text-to-Speech3 Apr
  4. 2020DiffWave: A Versatile Diffusion Model for Audio Synthesis21 Sep

Other references · 36

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  2. 2020Denoising Diffusion Probabilistic Modelscited by 21,076
  3. 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,713
  4. 2020Score-Based Generative Modeling through Stochastic Differential Equationscited by 9,122
  5. 2021Diffusion Models Beat GANs on Image Synthesiscited by 8,255
  6. 2015Deep Unsupervised Learning using Nonequilibrium Thermodynamicscited by 7,058
  7. 2022Hierarchical Text-Conditional Image Generation with CLIP Latentscited by 6,585
  8. 2019Generative Modeling by Estimating Gradients of the Data Distributioncited by 4,101
  9. 2021GLIDE: Towards Photorealistic Image Generation and Editing with Text-Guided Diffusion Modelscited by 3,614
  10. 2016WaveNet: A Generative Model for Raw Audiocited by 1,722
  11. 2020HiFi-GAN: Generative Adversarial Networks for Efficient and High Fidelity Speech Synthesiscited by 1,154
  12. 2020Improved Techniques for Training Score-Based Generative Modelscited by 1,043
24 more
  1. 2018VoxCeleb2: Deep Speaker Recognitioncited by 875
  2. 2017Natural TTS Synthesis by Conditioning WaveNet on Mel Spectrogram Predictionscited by 804
  3. 2020FastSpeech 2: Fast and High-Quality End-to-End Text to Speechcited by 717
  4. 2019LibriTTS: A Corpus Derived from LibriSpeech for Text-to-Speechcited by 649
  5. 2021Conditional Variational Autoencoder with Adversarial Learning for End-to-End Text-to-Speechcited by 561
  6. 2017Tacotron: Towards End-to-End Speech Synthesiscited by 490
  7. 2019Libri-Light: A Benchmark for ASR with Limited or No Supervisioncited by 467
  8. 2018WaveGlow: A Flow-based Generative Network for Speech Synthesiscited by 309
  9. 2017Generalized End-to-End Loss for Speaker Verificationcited by 293
  10. 2018Transfer Learning from Speaker Verification to Multispeaker Text-To-Speech Synthesiscited by 282
  11. 2021YourTTS: Towards Zero-Shot Multi-Speaker TTS and Zero-Shot Voice Conversion for everyonecited by 278
  12. 2020Glow-TTS: A Generative Flow for Text-to-Speech via Monotonic Alignment Searchcited by 273
  13. 2019NeMo: a toolkit for building AI applications using Neural Modulescited by 235
  14. 2018Adversarial Audio Synthesiscited by 225
  15. 2021AdaSpeech: Adaptive Text to Speech for Custom Voicecited by 98
  16. 2021Meta-StyleSpeech : Multi-Speaker Adaptive Text-to-Speech Generationcited by 93
  17. 2020End-to-End Adversarial Text-to-Speechcited by 77
  18. 2019Zero-Shot Multi-Speaker Text-To-Speech with State-of-the-art Neural Speaker Embeddingscited by 73
  19. 2021WaveGrad 2: Iterative Refinement for Text-to-Speech Synthesiscited by 65
  20. 2021SC-GlowTTS: an Efficient Zero-Shot Multi-Speaker Text-To-Speech Modelcited by 56
  21. 2018FloWaveNet : A Generative Flow for Raw Audiocited by 54
  22. 2018Sample Efficient Adaptive Text-to-Speechcited by 45
  23. 2022AdaSpeech 4: Adaptive Text to Speech in Zero-Shot Scenarioscited by 41
  24. 2021AdaSpeech 2: Adaptive Text to Speech with Untranscribed Datacited by 17

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2022: 4; 2023: 17; 2024: 9; 2025: 4; 2026: 1.

35 papers.

Sort:

By area:

  1. 2022Diffusion Models: A Comprehensive Survey of Methods and Applicationscited by 1,114
  2. 2022DiffusionDet: Diffusion Model for Object Detectioncited by 354
  3. 2022A Survey on Generative Diffusion Modelcited by 240
  4. 2023Diffusion-Based 3D Human Pose Estimation with Multi-Hypothesis Aggregationcited by 82
  5. 2023A Survey on Audio Diffusion Models: Text To Speech Synthesis and Enhancement in Generative AIcited by 68
  6. 2022Solving Audio Inverse Problems with a Diffusion Modelcited by 62
  7. 2023Efficient Neural Music Generationcited by 50
  8. 2023DDDM-VC: Decoupled Denoising Diffusion Models with Disentangled Representation and Prior Mixup for Verified Robust Voice Conversioncited by 38
  9. 2024Foundation Models for Music: A Surveycited by 37
  10. 2023HierSpeech++: Bridging the Gap between Semantic and Acoustic Representation of Speech by Hierarchical Variational Inference for Zero-shot Speech Synthesiscited by 35
  11. 2024Stochastic Runge-Kutta Methods: Provable Acceleration of Diffusion Modelscited by 35
  12. 2023Speech Driven Video Editing via an Audio-Conditioned Diffusion Modelcited by 28
23 more
  1. 2023Imaginary Voice: Face-styled Diffusion Model for Text-to-Speechcited by 27
  2. 2023DiffTAD: Temporal Action Detection with Proposal Denoising Diffusioncited by 19
  3. 2023The Ethical Implications of Generative Audio Models: A Systematic Literature Reviewcited by 18
  4. 2023UnitSpeech: Speaker-adaptive Speech Synthesis with Untranscribed Datacited by 15
  5. 2023ZET-Speech: Zero-shot adaptive Emotion-controllable Text-to-Speech Synthesis with Diffusion and Style-based Modelscited by 14
  6. 2024CM-TTS: Enhancing Real Time Text-to-Speech Synthesis Efficiency through Weighted Samplers and Consistency Modelscited by 13
  7. 2024Towards Understanding Unsafe Video Generationcited by 10
  8. 2023HierVST: Hierarchical Adaptive Zero-shot Voice Style Transfercited by 9
  9. 2024Fake Artificial Intelligence Generated Contents (FAIGC): A Survey of Theories, Detection Methods, and Opportunitiescited by 8
  10. 2024A Survey of Deep Learning Audio Generation Methodscited by 5
  11. 2025Voice Cloning: Comprehensive Surveycited by 4
  12. 2026X-Voice: Enabling Everyone to Speak 30 Languages via Zero-Shot Cross-Lingual Voice Cloningcited by 4
  13. 2024On the Semantic Latent Space of Diffusion-Based Text-to-Speech Modelscited by 3
  14. 2024MakeSinger: A Semi-Supervised Training Method for Data-Efficient Singing Voice Synthesis via Classifier-free Diffusion Guidancecited by 3
  15. 2024Towards Zero-Shot Text-To-Speech for Arabic Dialectscited by 3
  16. 2023DiCLET-TTS: Diffusion Model based Cross-lingual Emotion Transfer for Text-to-Speech -- A Study between English and Mandarincited by 2
  17. 2023Enhance audio generation controllability through representation similarity regularizationcited by 2
  18. 2023A Comparative Study of Voice Conversion Models with Large-Scale Speech and Singing Data: The T13 Systems for the Singing Voice Conversion Challenge 2023cited by 2
  19. 2023Face-StyleSpeech: Enhancing Zero-shot Speech Synthesis from Face Images with Improved Face-to-Speech Mappingcited by 2
  20. 2025SupertonicTTS: Towards Highly Efficient and Streamlined Text-to-Speech Systemcited by 2
  21. 2025Audio Generation Through Score-Based Generative Modeling: Design Principles and Implementationcited by 1
  22. 2023An analysis on the effects of speaker embedding choice in non auto-regressive TTScited by 0
  23. 2025Generalized Visual Relation Detection with Diffusion Modelscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2021Guided-TTS: A Diffusion Model for Text-to-Speech via Classifier Guidance
  2. 2023Multi-GradSpeech: Towards Diffusion-based Multi-Speaker Text-to-speech Using Consistent Diffusion Models
  3. 2024DiTTo-TTS: Diffusion Transformers for Scalable Text-to-Speech without Domain-Specific Factors
  4. 2024DualSpeech: Enhancing Speaker-Fidelity and Text-Intelligibility Through Dual Classifier-Free Guidance
  5. 2025Posterior Transition Modeling for Unsupervised Diffusion-Based Speech Enhancement
  6. 2026Repurposing a Speech Classifier for Guided Diffusion-Based Speech Generation
  7. 2026LongCat-AudioDiT: High-Fidelity Diffusion Text-to-Speech in the Waveform Latent Space
  8. 2022Grad-StyleSpeech: Any-speaker Adaptive Text-to-Speech Synthesis with Diffusion Models