mlsanityData to 7 Oct 2026

DiffStyleTTS: Diffusion-based Hierarchical Prosody Modeling for Text-to-Speech with Diverse and Controllable Styles

arXiv 2412.03388 ↗First posted 4 Dec 2024cs.SD cs.AI cs.CLcited by 5

Not on the map. Why?

Abstract (arXiv)

Human speech exhibits rich and flexible prosodic variations. To address the one-to-many mapping problem from text to prosody in a reasonable and flexible manner, we propose DiffStyleTTS, a multi-speaker acoustic model based on a conditional diffusion module and an improved classifier-free guidance, which hierarchically models speech prosodic features, and controls different prosodic styles to guide prosody prediction. Experiments show that our method outperforms all baselines in naturalness and achieves superior synthesis speed compared to three diffusion-based baselines. Additionally, by adjusting the guiding scale, DiffStyleTTS effectively controls the guidance intensity of the synthetic prosody.

Key facts. Cited by 5 papers in the map (data through 7 Oct 2026). Builds on 18 of its 21 references that are in the map. First posted on arXiv 4 Dec 2024.

Builds on

18 of 21 references.

References · 18

  1. 2021Diffusion Models Beat GANs on Image Synthesiscited by 8,185
  2. 2022Classifier-Free Diffusion Guidancecited by 5,717
  3. 2021Improved Denoising Diffusion Probabilistic Modelscited by 3,736
  4. 2016WaveNet: A Generative Model for Raw Audiocited by 1,710
  5. 2020DiffWave: A Versatile Diffusion Model for Audio Synthesiscited by 1,339
  6. 2020HiFi-GAN: Generative Adversarial Networks for Efficient and High Fidelity Speech Synthesiscited by 1,139
  7. 2017Natural TTS Synthesis by Conditioning WaveNet on Mel Spectrogram Predictionscited by 800
  8. 2020FastSpeech 2: Fast and High-Quality End-to-End Text to Speechcited by 710
  9. 2017Tacotron: Towards End-to-End Speech Synthesiscited by 487
  10. 2021Grad-TTS: A Diffusion Probabilistic Model for Text-to-Speechcited by 445
  11. 2023Common Diffusion Noise Schedules and Sample Steps are Flawedcited by 308
  12. 2018Style Tokens: Unsupervised Style Modeling, Control and Transfer in End-to-End Speech Synthesiscited by 257
6 more
  1. 2021Diff-TTS: A Denoising Diffusion Model for Text-to-Speechcited by 156
  2. 2021DiffSinger: Singing Voice Synthesis via Shallow Diffusion Mechanismcited by 154
  3. 2022ProDiff: Progressive Fast Diffusion Model For High-Quality Text-to-Speechcited by 147
  4. 2021Guided-TTS: A Diffusion Model for Text-to-Speech via Classifier Guidancecited by 84
  5. 2023CoMoSpeech: One-Step Speech and Singing Voice Synthesis via Consistency Modelcited by 44
  6. 2023DiffProsody: Diffusion-based Latent Prosody Generation for Expressive Speech Synthesis with Prosody Conditional Adversarial Trainingcited by 15

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 1; 2025: 3; 2026: 1.

5 papers.

  1. 2024Towards Controllable Speech Synthesis in the Era of Large Language Models: A Systematic Surveycited by 22
  2. 2025CoMelSinger: Discrete Token-Based Zero-Shot Singing Synthesis With Structured Melody Control and Guidancecited by 3
  3. 2025UDDETTS: Unifying Discrete and Dimensional Emotions for Controllable Emotional Text-to-Speechcited by 2
  4. 2025HiStyle: Hierarchical Style Embedding Predictor for Text-Prompt-Guided Controllable Speech Synthesiscited by 1
  5. 2026VocalNet-MDM: Accelerating Streaming Speech LLM via Self-Distilled Masked Diffusion Modelingcited by 1