mlsanityData to 7 Oct 2026

EmoSphere-TTS: Emotional Style and Intensity Modeling via Spherical Emotion Vector for Controllable Emotional Text-to-Speech

arXiv 2406.07803 ↗First posted 12 Jun 2024cs.SD cs.AI eess.AScited by 23

Not on the map. Why?

Abstract (arXiv)

Despite rapid advances in the field of emotional text-to-speech (TTS), recent studies primarily focus on mimicking the average style of a particular emotion. As a result, the ability to manipulate speech emotion remains constrained to several predefined labels, compromising the ability to reflect the nuanced variations of emotion. In this paper, we propose EmoSphere-TTS, which synthesizes expressive emotional speech by using a spherical emotion vector to control the emotional style and intensity of the synthetic speech. Without any human annotation, we use the arousal, valence, and dominance pseudo-labels to model the complex nature of emotion via a Cartesian-spherical transformation. Furthermore, we propose a dual conditional adversarial network to improve the quality of generated speech by reflecting the multi-aspect characteristics. The experimental results demonstrate the model ability to control emotional style and intensity with high-quality expressive speech.

Key facts. Cited by 23 papers in the map (data through 7 Oct 2026). Builds on 26 of its 37 references that are in the map. First posted on arXiv 12 Jun 2024.

Builds on

26 of 37 references.

References · 26

  1. 2016Layer Normalizationcited by 5,579
  2. 2022Robust Speech Recognition via Large-Scale Weak Supervisioncited by 3,972
  3. 2020wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representationscited by 3,853
  4. 2020FastSpeech 2: Fast and High-Quality End-to-End Text to Speechcited by 710
  5. 2017Tacotron: Towards End-to-End Speech Synthesiscited by 487
  6. 2022UTMOS: UTokyo-SaruLab System for VoiceMOS Challenge 2022cited by 381
  7. 2022BigVGAN: A Universal Neural Vocoder with Large-Scale Trainingcited by 245
  8. 2022Dawn of the transformer era in speech emotion recognition: closing the valence gapcited by 155
  9. 2021Emotional Voice Conversion: Theory, Databases and ESDcited by 113
  10. 2022Emotion Intensity and its Control for Emotional Voice Conversioncited by 35
  11. 2023PromptStyle: Controllable Style Transfer for Text-to-Speech with Natural Language Descriptionscited by 33
  12. 2022Speech Synthesis with Mixed Emotionscited by 28
14 more
  1. 2019Semi-Supervised Generative Modeling for Controllable Speech Synthesiscited by 21
  2. 2020The ins and outs of speaker recognition: lessons from VoxSRC 2020cited by 21
  3. 2021Cross-speaker emotion disentangling and transfer for end-to-end speech synthesiscited by 21
  4. 2021GANSpeech: Adversarial Training for High-Fidelity Multi-Speaker Speech Synthesiscited by 18
  5. 2022SyntaSpeech: Syntax-Aware Generative Adversarial Text-to-Speechcited by 18
  6. 2023DiffProsody: Diffusion-based Latent Prosody Generation for Expressive Speech Synthesis with Prosody Conditional Adversarial Trainingcited by 15
  7. 2023ZET-Speech: Zero-shot adaptive Emotion-controllable Text-to-Speech Synthesis with Diffusion and Style-based Modelscited by 14
  8. 2021Emotional Prosody Control for Speech Generationcited by 12
  9. 2022Language Model-Based Emotion Prediction Methods for Emotional Speech Synthesis Systemscited by 9
  10. 2023HierVST: Hierarchical Adaptive Zero-shot Voice Style Transfercited by 9
  11. 2022Cross-speaker Emotion Transfer Based On Prosody Compensation for End-to-End Speech Synthesiscited by 8
  12. 2022Text-driven Emotional Style Control and Cross-speaker Style Transfer in Neural TTScited by 6
  13. 2022Explicit Intensity Control for Accented Text-to-speechcited by 5
  14. 2023Learning Emotional Representations from Imbalanced Speech Data for Speech Emotion Recognition and Emotional Text-to-Speechcited by 4

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 4; 2025: 11; 2026: 8.

23 papers.

Sort:

By area:

  1. 2024Laugh Now Cry Later: Controlling Time-Varying Emotional States of Flow-Matching-Based Zero-Shot Text-to-Speechcited by 19
  2. 2024EmoSphere++: Emotion-Controllable Zero-Shot Text-to-Speech via Emotion-Adaptive Spherical Vectorcited by 19
  3. 2025CapSpeech: Enabling Downstream Applications in Style-Captioned Text-to-Speechcited by 11
  4. 2024Emotional Dimension Control in Language Model-Based Text-to-Speech: Spanning a Broad Spectrum of Human Emotionscited by 8
  5. 2026CoCoEmo: Composable and Controllable Human-Like Emotional TTS via Activation Steeringcited by 7
  6. 2025DiEmo-TTS: Disentangled Emotion Representations via Self-Supervised Distillation for Cross-Speaker Emotion Transfer in Text-to-Speechcited by 5
  7. 2026EmoShift: Lightweight Activation Steering for Enhanced Emotion-Aware Speech Synthesiscited by 5
  8. 2024EmoReg: Directional Latent Vector Modeling for Emotional Intensity Regularization in Diffusion-based Voice Conversioncited by 2
  9. 2025UDDETTS: Unifying Discrete and Dimensional Emotions for Controllable Emotional Text-to-Speechcited by 2
  10. 2025Beyond Global Emotion: Fine-Grained Emotional Speech Synthesis with Dynamic Word-Level Modulationcited by 2
  11. 2026Neuron-Level Emotion Control in Speech-Generative Large Audio-Language Modelscited by 2
  12. 2026EmoTra-TTS: Smooth Intra-Utterance Emotion Transitions for Speech Synthesiscited by 2
11 more
  1. 2025Spotlight-TTS: Spotlighting the Style via Voiced-Aware Style Extraction and Style Direction Adjustment for Expressive Text-to-Speechcited by 1
  2. 2025RLAIF-SPA: Structured AI Feedback for Semantic-Prosodic Alignment in Speech Synthesiscited by 1
  3. 2026HPRO: Hierarchical Progressive Reward Optimization via Preference Extraction for Emotional Text-to-Speechcited by 1
  4. 2025EmoSphere-SER: Enhancing Speech Emotion Recognition Through Spherical Representation with Auxiliary Classificationcited by 0
  5. 2025GamerAstra: Supporting 2D Non-Twitch Video Games for Blind and Low-Vision Players through a Multi-Agent Frameworkcited by 0
  6. 2025Emotion-Aware Speech Generation with Character-Specific Voices for Comicscited by 0
  7. 2025Cross-modal Consistency Guidance for Robust Emotion Control in Auto-Regressive TTS Modelscited by 0
  8. 2025Voiced-Aware Style Extraction and Style Direction Adjustment for Expressive Text-to-Speechcited by 0
  9. 2026DUET: Unified Dual-Space Emotion Control for Diffusion and Flow-Matching Driven Text-to-Speechcited by 0
  10. 2026Diagnose, Then Refine: A Closed-Loop TTS System with AudioLLM-Guided Correctioncited by 0
  11. 2026Alignment-Free Text-Audiobox for Voice Dubbing and Full-Duplex Dialogue Synthesiscited by 0