mlsanityData to 14 Oct 2026

EmoDiff: Intensity Controllable Emotional Text-to-Speech with Soft-Label Guidance

arXiv 2211.09496 ↗First posted 17 Nov 2022eess.AS cs.AI cs.HCcited by 37

Not on the map. Why?

Abstract (arXiv)

Although current neural text-to-speech (TTS) models are able to generate high-quality speech, intensity controllable emotional TTS is still a challenging task. Most existing methods need external optimizations for intensity calculation, leading to suboptimal results or degraded quality. In this paper, we propose EmoDiff, a diffusion-based TTS model where emotion intensity can be manipulated by a proposed soft-label guidance technique derived from classifier guidance. Specifically, instead of being guided with a one-hot vector for the specified emotion, EmoDiff is guided with a soft label where the value of the specified emotion and textitNeutral is set to α and 1-α respectively. The α here represents the emotion intensity and can be chosen from 0 to 1. Our experiments show that EmoDiff can precisely control the emotion intensity while maintaining high voice quality. Moreover, diverse speech with specified emotion intensity can be generated by sampling in the reverse denoising process.

Key facts. Cited by 37 papers in the map (data through 14 Oct 2026). Builds on 27 of its 33 references that are in the map. First posted on arXiv 17 Nov 2022.

Builds on

27 of 33 references.

References · 27

  1. 2020Denoising Diffusion Probabilistic Modelscited by 21,076
  2. 2020Score-Based Generative Modeling through Stochastic Differential Equationscited by 9,122
  3. 2021Diffusion Models Beat GANs on Image Synthesiscited by 8,255
  4. 2019Generative Modeling by Estimating Gradients of the Data Distributioncited by 4,101
  5. 2020DiffWave: A Versatile Diffusion Model for Audio Synthesiscited by 1,354
  6. 2020HiFi-GAN: Generative Adversarial Networks for Efficient and High Fidelity Speech Synthesiscited by 1,154
  7. 2020WaveGrad: Estimating Gradients for Waveform Generationcited by 622
  8. 2021Conditional Variational Autoencoder with Adversarial Learning for End-to-End Text-to-Speechcited by 561
  9. 2021Grad-TTS: A Diffusion Probabilistic Model for Text-to-Speechcited by 447
  10. 2021More Control for Free! Image Synthesis with Semantic Diffusion Guidancecited by 212
  11. 2021Diff-TTS: A Denoising Diffusion Model for Text-to-Speechcited by 157
  12. 2021DiffSinger: Singing Voice Synthesis via Shallow Diffusion Mechanismcited by 154
15 more
  1. 2022FastDiff: A Fast Conditional Diffusion Model for High-Quality Speech Synthesiscited by 136
  2. 2020Seen and Unseen emotional style transfer for voice conversion with a new emotional speech datasetcited by 121
  3. 2021Guided-TTS: A Diffusion Model for Text-to-Speech via Classifier Guidancecited by 84
  4. 2022BDDM: Bilateral Denoising Diffusion Models for Fast and High-Quality Speech Synthesiscited by 64
  5. 2020Fully-hierarchical fine-grained prosody modeling for interpretable speech synthesiscited by 41
  6. 2022Emotion Intensity and its Control for Emotional Voice Conversioncited by 35
  7. 2022VQTTS: High-Fidelity Text-to-Speech Synthesis with Self-Supervised VQ Acoustic Featurecited by 32
  8. 2017Emotional End-to-End Neural Speech Synthesizercited by 30
  9. 2020Fine-grained Emotion Strength Transfer, Control and Prediction for Emotional Speech Synthesiscited by 29
  10. 2022MsEmoTTS: Multi-scale emotion transfer, prediction, and control for emotional speech synthesiscited by 29
  11. 2019Emotional speech synthesis with rich and granularized controlcited by 28
  12. 2022Speech Synthesis with Mixed Emotionscited by 28
  13. 2021Expressive Text-to-Speech using Style Tagcited by 20
  14. 2021Phone-Level Prosody Modelling with GMM-Based MDN for Diverse and Controllable Speech Synthesiscited by 10
  15. 2022Unsupervised word-level prosody tagging for controllable speech synthesiscited by 7

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 13; 2024: 11; 2025: 10; 2026: 3.

37 papers.

Sort:

By area:

  1. 2023Matcha-TTS: A fast TTS architecture with conditional flow matchingcited by 145
  2. 2023A Survey on Audio Diffusion Models: Text To Speech Synthesis and Enhancement in Generative AIcited by 68
  3. 2023VoiceFlow: Efficient Text-to-Speech with Rectified Flow Matchingcited by 49
  4. 2025EmoVoice: LLM-based Emotional Text-To-Speech Model with Freestyle Text Promptingcited by 34
  5. 2023EmoMix: Emotion Mixing via Diffusion Models for Emotional Speech Synthesiscited by 19
  6. 2024Laugh Now Cry Later: Controlling Time-Varying Emotional States of Flow-Matching-Based Zero-Shot Text-to-Speechcited by 19
  7. 2023EmoSpeech: Guiding FastSpeech2 Towards Emotional Text to Speechcited by 15
  8. 2023ZET-Speech: Zero-shot adaptive Emotion-controllable Text-to-Speech Synthesis with Diffusion and Style-based Modelscited by 14
  9. 2023Leveraging Speech PTM, Text LLM, and Emotional TTS for Speech Emotion Recognitioncited by 12
  10. 2025CapSpeech: Enabling Downstream Applications in Style-Captioned Text-to-Speechcited by 11
  11. 2024ED-TTS: Multi-Scale Emotion Modeling using Cross-Domain Emotion Diarization for Emotional Speech Synthesiscited by 9
  12. 2023EMOCONV-DIFF: Diffusion-based Speech Emotion Conversion for Non-parallel and In-the-wild Datacited by 7
25 more
  1. 2023StyleCap: Automatic Speaking-Style Captioning from Speech Based on Speech and Language Self-supervised Learning Modelscited by 7
  2. 2024Rasa: Building Expressive Speech Synthesis Systems for Indian Languages in Low-resource Settingscited by 7
  3. 2024A Review of Human Emotion Synthesis Based on Generative Technologycited by 7
  4. 2026EmoShift: Lightweight Activation Steering for Enhanced Emotion-Aware Speech Synthesiscited by 7
  5. 2023DSE-TTS: Dual Speaker Embedding for Cross-Lingual Text-to-Speechcited by 5
  6. 2024UMETTS: A Unified Framework for Emotional Text-to-Speech Synthesis with Multimodal Promptscited by 5
  7. 2025Can Emotion Fool Anti-spoofing?cited by 4
  8. 2025Prompt-Unseen-Emotion: Zero-shot Expressive Speech Synthesis with Prompt-LLM Contextual Knowledge for Mixed Emotionscited by 4
  9. 2025RRPO: Robust Reward Policy Optimization for LLM-based Emotional TTScited by 4
  10. 2024On the Semantic Latent Space of Diffusion-Based Text-to-Speech Modelscited by 3
  11. 2025Task Vector in TTS: Toward Emotionally Expressive Dialectal Speech Synthesiscited by 3
  12. 2024AUTODIFF: Autoregressive Diffusion Modeling for Structure-based Drug Designcited by 2
  13. 2024Facial Expression-Enhanced TTS: Combining Face Representation and Emotion Intensity for Adaptive Speechcited by 2
  14. 2025UDDETTS: Unifying Discrete and Dimensional Emotions for Controllable Emotional Text-to-Speechcited by 2
  15. 2025Emotion-Aligned Generation in Diffusion Text to Speech Models via Preference-Guided Optimizationcited by 2
  16. 2023AffectEcho: Speaker Independent and Language-Agnostic Emotion and Affect Transfer for Speech Synthesiscited by 1
  17. 2023Expressive TTS Driven by Natural Language Prompts Using Few Human Annotationscited by 1
  18. 2024Humane Speech Synthesis through Zero-Shot Emotion and Disfluency Generationcited by 1
  19. 2024EmoSpeech: A Corpus of Emotionally Rich and Contextually Detailed Speech Annotationscited by 1
  20. 2025Shushing! Let's Imagine an Authentic Speech from the Silent Videocited by 1
  21. 2025RLAIF-SPA: Structured AI Feedback for Semantic-Prosodic Alignment in Speech Synthesiscited by 1
  22. 2023CASEIN: Cascading Explicit and Implicit Control for Fine-grained Emotion Intensity Regulationcited by 0
  23. 2024GTR-Voice: Articulatory Phonetics Informed Controllable Expressive Speech Synthesiscited by 0
  24. 2026DUET: Unified Dual-Space Emotion Control for Diffusion and Flow-Matching Driven Text-to-Speechcited by 0
  25. 2026UniSAE: Unified Speech Attribute Editing on Speaker, Emotion and Low-Level Content via Discrete Phonetic Posteriorgram Modellingcited by 0