mlsanityData to 14 Oct 2026

EmoSpeech: Guiding FastSpeech2 Towards Emotional Text to Speech

arXiv 2307.00024 ↗First posted 28 Jun 2023eess.AS cs.LG cs.SDcited by 15

Not on the map. Why?

Abstract (arXiv)

State-of-the-art speech synthesis models try to get as close as possible to the human voice. Hence, modelling emotions is an essential part of Text-To-Speech (TTS) research. In our work, we selected FastSpeech2 as the starting point and proposed a series of modifications for synthesizing emotional speech. According to automatic and human evaluation, our model, EmoSpeech, surpasses existing models regarding both MOS score and emotion recognition accuracy in generated speech. We provided a detailed ablation study for every extension to FastSpeech2 architecture that forms EmoSpeech. The uneven distribution of emotions in the text is crucial for better, synthesized speech and intonation perception. Our model includes a conditioning mechanism that effectively handles this issue by allowing emotions to contribute to each phone with varying intensity levels. The human assessment indicates that proposed modifications generate audio with higher MOS and emotional expressiveness.

Key facts. Cited by 15 papers in the map (data through 14 Oct 2026). Builds on 26 of its 35 references that are in the map. First posted on arXiv 28 Jun 2023.

Builds on

26 of 35 references.

References · 26

  1. 2017Attention Is All You Needcited by 60,361
  2. 2014Adam: A Method for Stochastic Optimizationcited by 37,240
  3. 2020Denoising Diffusion Probabilistic Modelscited by 21,076
  4. 2016Layer Normalizationcited by 5,669
  5. 2015Variational Inference with Normalizing Flowscited by 2,170
  6. 2017Natural TTS Synthesis by Conditioning WaveNet on Mel Spectrogram Predictionscited by 804
  7. 2020FastSpeech 2: Fast and High-Quality End-to-End Text to Speechcited by 717
  8. 2021Conditional Variational Autoencoder with Adversarial Learning for End-to-End Text-to-Speechcited by 561
  9. 2021Grad-TTS: A Diffusion Probabilistic Model for Text-to-Speechcited by 447
  10. 2020Glow-TTS: A Generative Flow for Text-to-Speech via Monotonic Alignment Searchcited by 273
  11. 2018Style Tokens: Unsupervised Style Modeling, Control and Transfer in End-to-End Speech Synthesiscited by 259
  12. 2022NaturalSpeech: End-to-End Text to Speech Synthesis with Human-Level Qualitycited by 152
14 more
  1. 2021Emotional Voice Conversion: Theory, Databases and ESDcited by 115
  2. 2022PromptTTS: Controllable Text-to-Speech with Text Descriptionscited by 110
  3. 2021AdaSpeech: Adaptive Text to Speech for Custom Voicecited by 98
  4. 2023InstructTTS: Modelling Expressive TTS in Discrete Latent Space with Natural Language Style Promptcited by 92
  5. 2022AdaSpeech 4: Adaptive Text to Speech in Zero-Shot Scenarioscited by 41
  6. 2022EmoDiff: Intensity Controllable Emotional Text-to-Speech with Soft-Label Guidancecited by 37
  7. 2021Deep Learning Based Assessment of Synthetic Speech Naturalnesscited by 35
  8. 2020Controllable Emotion Transfer For End-to-End Speech Synthesiscited by 32
  9. 2022iSTFTNet: Fast and Lightweight Mel-Spectrogram Vocoder Incorporating Inverse Short-Time Fourier Transformcited by 32
  10. 2017Emotional End-to-End Neural Speech Synthesizercited by 30
  11. 2019Emotional speech synthesis with rich and granularized controlcited by 28
  12. 2020VocGAN: A High-Fidelity Real-time Vocoder with a Hierarchically-nested Adversarial Networkcited by 26
  13. 2020Emotion controllable speech synthesis using emotion-unlabeled dataset with the assistance of cross-domain speech emotion recognitioncited by 19
  14. 2021GANSpeech: Adversarial Training for High-Fidelity Multi-Speaker Speech Synthesiscited by 18

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 4; 2025: 8; 2026: 3.

15 papers.

Sort:

By area:

  1. 2025EmoVoice: LLM-based Emotional Text-To-Speech Model with Freestyle Text Promptingcited by 34
  2. 2024A Comprehensive Survey of Direct Preference Optimization: Datasets, Theories, Variants, and Applicationscited by 27
  3. 2024Controlling Emotion in Text-to-Speech with Natural Language Promptscited by 9
  4. 2024A Review of Human Emotion Synthesis Based on Generative Technologycited by 7
  5. 2024UMETTS: A Unified Framework for Emotional Text-to-Speech Synthesis with Multimodal Promptscited by 5
  6. 2025Prompt-Unseen-Emotion: Zero-shot Expressive Speech Synthesis with Prompt-LLM Contextual Knowledge for Mixed Emotionscited by 4
  7. 2025Marco-Voice Technical Reportcited by 3
  8. 2025Beyond Global Emotion: Fine-Grained Emotional Speech Synthesis with Dynamic Word-Level Modulationcited by 3
  9. 2025Task Vector in TTS: Toward Emotionally Expressive Dialectal Speech Synthesiscited by 3
  10. 2025MultiActor-Audiobook: Zero-Shot Audiobook Generation with Faces and Voices of Multiple Speakerscited by 2
  11. 2025Emotion-Aligned Generation in Diffusion Text to Speech Models via Preference-Guided Optimizationcited by 2
  12. 2025RLAIF-SPA: Structured AI Feedback for Semantic-Prosodic Alignment in Speech Synthesiscited by 1
3 more
  1. 2026AgentSteerTTS: A Multi-Agent Closed-Loop Framework for Composite-Instruction Text-to-Speechcited by 1
  2. 2026Causal Prosody Mediation for Text-to-Speech:Counterfactual Training of Duration, Pitch, and Energy in FastSpeech2cited by 0
  3. 2026Iterative Self-Learning for Expressive Text-to-Speech Synthesiscited by 0