mlsanityData to 14 Oct 2026

ZET-Speech: Zero-shot adaptive Emotion-controllable Text-to-Speech Synthesis with Diffusion and Style-based Models

arXiv 2305.13831 ↗First posted 23 May 2023cs.SD cs.CL eess.AScited by 14

Not on the map. Why?

Abstract (arXiv)

Emotional Text-To-Speech (TTS) is an important task in the development of systems (e.g., human-like dialogue agents) that require natural and emotional speech. Existing approaches, however, only aim to produce emotional TTS for seen speakers during training, without consideration of the generalization to unseen speakers. In this paper, we propose ZET-Speech, a zero-shot adaptive emotion-controllable TTS model that allows users to synthesize any speaker's emotional speech using only a short, neutral speech segment and the target emotion label. Specifically, to enable a zero-shot adaptive TTS model to synthesize emotional speech, we propose domain adversarial learning and guidance methods on the diffusion model. Experimental results demonstrate that ZET-Speech successfully synthesizes natural and emotional speech with the desired emotion for both seen and unseen speakers. Samples are at https://ZET-Speech.github.io/ZET-Speech-Demo/.

Key facts. Cited by 14 papers in the map (data through 14 Oct 2026). Builds on 28 of its 31 references that are in the map. First posted on arXiv 23 May 2023.

Builds on

28 of 31 references.

References · 28

  1. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,863
  2. 2020Score-Based Generative Modeling through Stochastic Differential Equationscited by 9,122
  3. 2021Diffusion Models Beat GANs on Image Synthesiscited by 8,255
  4. 2018A Style-Based Generator Architecture for Generative Adversarial Networkscited by 6,013
  5. 2022Classifier-Free Diffusion Guidancecited by 5,847
  6. 2022Robust Speech Recognition via Large-Scale Weak Supervisioncited by 4,057
  7. 2015Domain-Adversarial Training of Neural Networkscited by 3,044
  8. 2020HiFi-GAN: Generative Adversarial Networks for Efficient and High Fidelity Speech Synthesiscited by 1,154
  9. 2023Neural Codec Language Models are Zero-Shot Text to Speech Synthesizerscited by 752
  10. 2019LibriTTS: A Corpus Derived from LibriSpeech for Text-to-Speechcited by 649
  11. 2021AST: Audio Spectrogram Transformercited by 643
  12. 2021Conditional Variational Autoencoder with Adversarial Learning for End-to-End Text-to-Speechcited by 561
16 more
  1. 2021Grad-TTS: A Diffusion Probabilistic Model for Text-to-Speechcited by 447
  2. 2018Style Tokens: Unsupervised Style Modeling, Control and Transfer in End-to-End Speech Synthesiscited by 259
  3. 2021Emotional Voice Conversion: Theory, Databases and ESDcited by 115
  4. 2018Neural Voice Cloning with a Few Samplescited by 100
  5. 2021AdaSpeech: Adaptive Text to Speech for Custom Voicecited by 98
  6. 2021Meta-StyleSpeech : Multi-Speaker Adaptive Text-to-Speech Generationcited by 93
  7. 2022AdaSpeech 4: Adaptive Text to Speech in Zero-Shot Scenarioscited by 41
  8. 2022EmoDiff: Intensity Controllable Emotional Text-to-Speech with Soft-Label Guidancecited by 37
  9. 2022Guided-TTS 2: A Diffusion Model for High-quality Adaptive Text-to-Speech with Untranscribed Datacited by 35
  10. 2022Grad-StyleSpeech: Any-speaker Adaptive Text-to-Speech Synthesis with Diffusion Modelscited by 34
  11. 2019Emotional speech synthesis with rich and granularized controlcited by 28
  12. 2022Speech Synthesis with Mixed Emotionscited by 28
  13. 2021Emotional Prosody Control for Speech Generationcited by 12
  14. 2021Cross-speaker Emotion Transfer Based on Speaker Condition Layer Normalization and Semi-Supervised Training in Text-To-Speechcited by 8
  15. 2022Cross-Speaker Emotion Transfer for Low-Resource Text-to-Speech Using Non-Parallel Voice Conversion with Pitch-Shift Data Augmentationcited by 8
  16. 2022Cross-speaker Emotion Transfer Based On Prosody Compensation for End-to-End Speech Synthesiscited by 8

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 1; 2024: 4; 2025: 8; 2026: 1.

14 papers.

Sort:

By area:

  1. 2023On the Design Fundamentals of Diffusion Models: A Surveycited by 60
  2. 2025EmoVoice: LLM-based Emotional Text-To-Speech Model with Freestyle Text Promptingcited by 34
  3. 2024EmoSphere-TTS: Emotional Style and Intensity Modeling via Spherical Emotion Vector for Controllable Emotional Text-to-Speechcited by 24
  4. 2024EmoKnob: Enhance Voice Cloning with Fine-Grained Emotion Controlcited by 13
  5. 2024A Review of Human Emotion Synthesis Based on Generative Technologycited by 7
  6. 2025Voice Cloning: Comprehensive Surveycited by 4
  7. 2025OZSpeech: One-step Zero-shot Speech Synthesis with Learned-Prior-Conditioned Flow Matchingcited by 4
  8. 2025RRPO: Robust Reward Policy Optimization for LLM-based Emotional TTScited by 4
  9. 2024Enhancing Emotional Text-to-Speech Controllability with Natural Language Guidance through Contrastive Learning and Diffusion Modelscited by 3
  10. 2025Voice Impression Control in Zero-Shot TTScited by 3
  11. 2025UDDETTS: Unifying Discrete and Dimensional Emotions for Controllable Emotional Text-to-Speechcited by 2
  12. 2026DiFlowDubber: Discrete Flow Matching for Automated Video Dubbing via Cross-Modal Alignment and Synchronizationcited by 2
2 more
  1. 2025RLAIF-SPA: Structured AI Feedback for Semantic-Prosodic Alignment in Speech Synthesiscited by 1
  2. 2025Flamed-TTS: Flow Matching Attention-Free Models for Efficient Generating and Dynamic Pacing Zero-shot Text-to-Speechcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024EmoSphere++: Emotion-Controllable Zero-Shot Text-to-Speech via Emotion-Adaptive Spherical Vector
  2. 2024Laugh Now Cry Later: Controlling Time-Varying Emotional States of Flow-Matching-Based Zero-Shot Text-to-Speech
  3. 2024StyleTTS-ZS: Efficient High-Quality Zero-Shot Text-to-Speech Synthesis with Distilled Time-Varying Style Diffusion
  4. 2024Hierarchical Control of Emotion Rendering in Speech Synthesis
  5. 2025TTS-CtrlNet: Time varying emotion aligned text-to-speech generation with ControlNet
  6. 2025Emotional Face-to-Speech
  7. 2025ZSVC: Zero-shot Style Voice Conversion with Disentangled Latent Diffusion Models and Adversarial Training
  8. 2026TRACE-EVC: Text-Guided Relative Affective Control for Zero-Shot Emotional Voice Conversion ↗