mlsanityData to 7 Oct 2026

DiffGAN-TTS: High-Fidelity and Efficient Text-to-Speech with Denoising Diffusion GANs

arXiv 2201.11972 ↗First posted 28 Jan 2022eess.AS cs.CL cs.SDcited by 43

On the map: Diffusion models → Audio, speech and music

Abstract (arXiv)

Denoising diffusion probabilistic models (DDPMs) are expressive generative models that have been used to solve a variety of speech synthesis problems. However, because of their high sampling costs, DDPMs are difficult to use in real-time speech processing applications. In this paper, we introduce DiffGAN-TTS, a novel DDPM-based text-to-speech (TTS) model achieving high-fidelity and efficient speech synthesis. DiffGAN-TTS is based on denoising diffusion generative adversarial networks (GANs), which adopt an adversarially-trained expressive model to approximate the denoising distribution. We show with multi-speaker TTS experiments that DiffGAN-TTS can generate high-fidelity speech samples within only 4 denoising steps. We present an active shallow diffusion mechanism to further speed up inference. A two-stage training scheme is proposed, with a basic TTS acoustic model trained at stage one providing valuable prior information for a DDPM trained at stage two. Our experiments show that DiffGAN-TTS can achieve high synthesis performance with only 1 denoising step.

Key facts. Cited by 43 papers in the map (data through 7 Oct 2026). Placed in Audio, speech and music, one of 14 directions in Diffusion models. Builds on 34 of its 53 references that are in the map. First posted on arXiv 28 Jan 2022.

Builds on

34 of 53 references.

Closest ancestors · 3

Same direction, newest first

  1. 2021Grad-TTS: A Diffusion Probabilistic Model for Text-to-Speech13 May
  2. 2021Symbolic Music Generation with Diffusion Models30 Mar
  3. 2020DiffWave: A Versatile Diffusion Model for Audio Synthesis21 Sep

Other references · 31

  1. 2017Attention Is All You Needcited by 59,932
  2. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  3. 2020Denoising Diffusion Probabilistic Modelscited by 20,851
  4. 2020Score-Based Generative Modeling through Stochastic Differential Equationscited by 8,964
  5. 2015Deep Unsupervised Learning using Nonequilibrium Thermodynamicscited by 7,005
  6. 2018A Style-Based Generator Architecture for Generative Adversarial Networkscited by 5,986
  7. 2016WaveNet: A Generative Model for Raw Audiocited by 1,710
  8. 2020HiFi-GAN: Generative Adversarial Networks for Efficient and High Fidelity Speech Synthesiscited by 1,139
  9. 2016Least Squares Generative Adversarial Networkscited by 1,000
  10. 2020FastSpeech 2: Fast and High-Quality End-to-End Text to Speechcited by 710
  11. 2020WaveGrad: Estimating Gradients for Waveform Generationcited by 620
  12. 2021Conditional Variational Autoencoder with Adversarial Learning for End-to-End Text-to-Speechcited by 555
19 more
  1. 2021Tackling the Generative Learning Trilemma with Denoising Diffusion GANscited by 515
  2. 2017Tacotron: Towards End-to-End Speech Synthesiscited by 487
  3. 2015Autoencoding beyond pixels using a learned similarity metriccited by 407
  4. 2019MelGAN: Generative Adversarial Networks for Conditional Waveform Synthesiscited by 351
  5. 2020Glow-TTS: A Generative Flow for Text-to-Speech via Monotonic Alignment Searchcited by 269
  6. 2020Permutation Invariant Graph Generation via Score-Based Generative Modelingcited by 263
  7. 2020End-to-End Adversarial Text-to-Speechcited by 77
  8. 2017Deep Voice 2: Multi-Speaker Neural Text-to-Speechcited by 75
  9. 2020Parallel Tacotron: Non-Autoregressive and Controllable TTScited by 49
  10. 2021One TTS Alignment To Rule Them Allcited by 47
  11. 2021PortaSpeech: Portable and High-Quality Generative Text-to-Speechcited by 47
  12. 2021MBNet: MOS Prediction for Synthesized Speech with Mean-Bias Networkcited by 37
  13. 2020VocGAN: A High-Fidelity Real-time Vocoder with a Hierarchically-nested Adversarial Networkcited by 26
  14. 2021LDNet: Unified Listener Dependent Modeling in MOS Prediction for Synthetic Speechcited by 24
  15. 2020AlignTTS: Efficient Feed-Forward Text-to-Speech System without Explicit Alignmentcited by 20
  16. 2021Utilizing Self-supervised Representations for MOS Predictioncited by 20
  17. 2021GANSpeech: Adversarial Training for High-Fidelity Multi-Speaker Speech Synthesiscited by 18
  18. 2020EfficientTTS: An Efficient and High-Quality Text-to-Speech Architecturecited by 14
  19. 2021VARA-TTS: Non-Autoregressive Text-to-Speech Synthesis based on Very Deep VAE with Residual Attentioncited by 12

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 7; 2023: 16; 2024: 10; 2025: 7; 2026: 3.

43 papers.

Sort:

By area:

  1. 2022LION: Latent Point Diffusion Models for 3D Shape Generationcited by 552
  2. 2023StyleTTS 2: Towards Human-Level Text-to-Speech through Style Diffusion and Adversarial Training with Large Speech Language Modelscited by 148
  3. 2022ProDiff: Progressive Fast Diffusion Model For High-Quality Text-to-Speechcited by 147
  4. 2022GENIE: Higher-Order Denoising Diffusion Solverscited by 127
  5. 2023A Survey on Audio Diffusion Models: Text To Speech Synthesis and Enhancement in Generative AIcited by 68
  6. 2023CoMoSpeech: One-Step Speech and Singing Voice Synthesis via Consistency Modelcited by 44
  7. 2024Autoregressive Diffusion Transformer for Text-to-Speech Synthesiscited by 34
  8. 2023Schrodinger Bridges Beat Diffusion Models on Text-to-Speech Synthesiscited by 29
  9. 2023FluentSpeech: Stutter-Oriented Automatic Speech Editing with Context-Aware Diffusion Modelscited by 23
  10. 2024SimpleSpeech: Towards Simple and Efficient Text-to-Speech with Scalar Latent Transformer Diffusion Modelscited by 23
  11. 2023DiffV2S: Diffusion-based Video-to-Speech Synthesis with Vision-guided Speaker Embeddingcited by 22
  12. 2023ReFlow-TTS: A Rectified Flow Model for High-fidelity Text-to-Speechcited by 22
31 more
  1. 2022ResGrad: Residual Denoising Diffusion Probabilistic Models for Text to Speechcited by 21
  2. 2022WaveFit: An Iterative and Non-autoregressive Neural Vocoder based on Fixed-Point Iterationcited by 20
  3. 2023The Ethical Implications of Generative Audio Models: A Systematic Literature Reviewcited by 18
  4. 2024CoVoMix: Advancing Zero-Shot Speech Generation for Human-like Multi-talker Conversationscited by 15
  5. 2023Star-Shaped Denoising Diffusion Probabilistic Modelscited by 13
  6. 2024CM-TTS: Enhancing Real Time Text-to-Speech Synthesis Efficiency through Weighted Samplers and Consistency Modelscited by 13
  7. 2024Diffusion Imitation from Observationcited by 12
  8. 2023DiffAR: Denoising Diffusion Autoregressive Model for Raw Speech Waveform Generationcited by 11
  9. 2023FrePolad: Frequency-Rectified Point Latent Diffusion for Point Cloud Generationcited by 9
  10. 2024Diffuse or Confuse: A Diffusion Deepfake Speech Datasetcited by 8
  11. 2025STOPA: A Database of Systematic VariaTion Of DeePfake Audio for Open-Set Source Tracing and Attributioncited by 6
  12. 2022Mandarin Singing Voice Synthesis with Denoising Diffusion Probabilistic Wasserstein GANcited by 5
  13. 2024DEX-TTS: Diffusion-based EXpressive Text-to-Speech with Style Modeling on Time Variabilitycited by 5
  14. 2024VoiceWukong: Benchmarking Deepfake Voice Detectioncited by 5
  15. 2023Boosting Fast and High-Quality Speech Synthesis with Linear Diffusioncited by 4
  16. 2024DiffSSD: A Diffusion-Based Dataset For Speech Forensicscited by 4
  17. 2025Voice Cloning: Comprehensive Surveycited by 4
  18. 2023Diverse and Expressive Speech Prosody Prediction with Denoising Diffusion Probabilistic Modelcited by 3
  19. 2024On the Semantic Latent Space of Diffusion-Based Text-to-Speech Modelscited by 3
  20. 2025SlimSpeech: Lightweight and Efficient Text-to-Speech with Slim Rectified Flowcited by 3
  21. 2025Continuous Autoregressive Modeling with Stochastic Monotonic Alignment for Speech Synthesiscited by 2
  22. 2023Adversarial Training of Denoising Diffusion Model Using Dual Discriminators for High-Fidelity Multi-Speaker TTScited by 1
  23. 2023Multi-GradSpeech: Towards Diffusion-based Multi-Speaker Text-to-speech Using Consistent Diffusion Modelscited by 1
  24. 2025Audio Generation Through Score-Based Generative Modeling: Design Principles and Implementationcited by 1
  25. 2025DiTReducio: A Training-Free Acceleration for DiT-Based TTS via Progressive Calibrationcited by 1
  26. 2026Quartet of Diffusions: Structure-Aware Point Cloud Generation through Part and Symmetry Guidancecited by 1
  27. 2022PSVRF: Learning to restore Pitch-Shifted Voice without referencecited by 0
  28. 2023Style Description based Text-to-Speech with Conditional Prosodic Layer Normalization based Diffusion GANcited by 0
  29. 2025Visually Grounded Narratives: Reducing Cognitive Burden in Researcher-Participant Interactioncited by 0
  30. 2026MambaVoiceCloning: Efficient and Expressive Text-to-Speech via State-Space Modeling and Diffusion Controlcited by 0
  31. 2026A survey of AI-generated voices and their detectioncited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2023Voice Conversion with Denoising Diffusion Probabilistic GAN Models
  2. 2021Diff-TTS: A Denoising Diffusion Model for Text-to-Speech
  3. 2023DiffVoice: Text-to-Speech with Latent Diffusion
  4. 2022FastDiff: A Fast Conditional Diffusion Model for High-Quality Speech Synthesis
  5. 2023DiffProsody: Diffusion-based Latent Prosody Generation for Expressive Speech Synthesis with Prosody Conditional Adversarial Training
  6. 2023U-DiT TTS: U-Diffusion Vision Transformer for Text-to-Speech
  7. 2025On the Design of Diffusion-based Neural Speech Codecs
  8. 2022BDDM: Bilateral Denoising Diffusion Models for Fast and High-Quality Speech Synthesis