mlsanityData to 7 Oct 2026

Emotional Prosody Control for Speech Generation

arXiv 2111.04730 ↗First posted 7 Nov 2021eess.AS cs.LG cs.SDcited by 12

Not on the map. Why?

Abstract (arXiv)

Machine-generated speech is characterized by its limited or unnatural emotional variation. Current text to speech systems generates speech with either a flat emotion, emotion selected from a predefined set, average variation learned from prosody sequences in training data or transferred from a source style. We propose a text to speech(TTS) system, where a user can choose the emotion of generated speech from a continuous and meaningful emotion space (Arousal-Valence space). The proposed TTS system can generate speech from the text in any speaker's style, with fine control of emotion. We show that the system works on emotion unseen during training and can scale to previously unseen speakers given his/her speech sample. Our work expands the horizon of the state-of-the-art FastSpeech2 backbone to a multi-speaker setting and gives it much-coveted continuous (and interpretable) affective control, without any observable degradation in the quality of the synthesized speech.

Key facts. Cited by 12 papers in the map (data through 7 Oct 2026). Builds on 19 of its 30 references that are in the map. First posted on arXiv 7 Nov 2021.

Builds on

19 of 30 references.

References · 19

  1. 2016WaveNet: A Generative Model for Raw Audiocited by 1,710
  2. 2017Natural TTS Synthesis by Conditioning WaveNet on Mel Spectrogram Predictionscited by 800
  3. 2017VoxCeleb: a large-scale speaker identification datasetcited by 714
  4. 2020FastSpeech 2: Fast and High-Quality End-to-End Text to Speechcited by 710
  5. 2017Tacotron: Towards End-to-End Speech Synthesiscited by 487
  6. 2019MelGAN: Generative Adversarial Networks for Conditional Waveform Synthesiscited by 351
  7. 2017Generalized End-to-End Loss for Speaker Verificationcited by 290
  8. 2018Transfer Learning from Speaker Verification to Multispeaker Text-To-Speech Synthesiscited by 280
  9. 2018Style Tokens: Unsupervised Style Modeling, Control and Transfer in End-to-End Speech Synthesiscited by 257
  10. 2017Parallel WaveNet: Fast High-Fidelity Speech Synthesiscited by 214
  11. 2018Towards End-to-End Prosody Transfer for Expressive Speech Synthesis with Tacotroncited by 171
  12. 2017Deep Voice: Real-time Neural Text-to-Speechcited by 83
7 more
  1. 2017Deep Voice 2: Multi-Speaker Neural Text-to-Speechcited by 75
  2. 2018ClariNet: Parallel Wave Generation in End-to-End Text-to-Speechcited by 70
  3. 2017Deep Voice 3: Scaling Text-to-Speech with Convolutional Sequence Learningcited by 67
  4. 2019Token-Level Ensemble Distillation for Grapheme-to-Phoneme Conversioncited by 26
  5. 2020CopyCat: Many-to-Many Fine-Grained Prosody Transfer for Neural Text-to-Speechcited by 22
  6. 2019Semi-Supervised Generative Modeling for Controllable Speech Synthesiscited by 21
  7. 2019Effective Use of Variational Embedding Capacity in Expressive End-to-End Speech Synthesiscited by 15

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 1; 2023: 4; 2024: 4; 2025: 2; 2026: 1.

12 papers.

  1. 2024EmoSphere-TTS: Emotional Style and Intensity Modeling via Spherical Emotion Vector for Controllable Emotional Text-to-Speechcited by 23
  2. 2024EmoSphere++: Emotion-Controllable Zero-Shot Text-to-Speech via Emotion-Adaptive Spherical Vectorcited by 19
  3. 2023ZET-Speech: Zero-shot adaptive Emotion-controllable Text-to-Speech Synthesis with Diffusion and Style-based Modelscited by 14
  4. 2023Fine-grained Emotional Control of Text-To-Speech: Learning To Rank Inter- And Intra-Class Emotion Intensitiescited by 5
  5. 2023ParrotTTS: Text-to-Speech synthesis by exploiting self-supervised representationscited by 4
  6. 2024Exploring speech style spaces with language models: Emotional TTS without emotion labelscited by 4
  7. 2023Prosody Analysis of Audiobookscited by 3
  8. 2022Controllable Accented Text-to-Speech Synthesiscited by 2
  9. 2025UDDETTS: Unifying Discrete and Dimensional Emotions for Controllable Emotional Text-to-Speechcited by 2
  10. 2025Fast-VGAN: Lightweight Voice Conversion with Explicit Control of F0 and Duration Parameterscited by 2
  11. 2024PRESENT: Zero-Shot Text-to-Prosody Controlcited by 0
  12. 2026Alignment-Free Text-Audiobox for Voice Dubbing and Full-Duplex Dialogue Synthesiscited by 0