mlsanityData to 19 Oct 2026

Emotional speech synthesis and recognition

259 papers1 this weekMedium confidence in this grouping: mostly one topic, with blurry edgesMedium confidence in this grouping: mostly one topic, with blurry edges · About

Keywords: emotion, emotional, speech, emotion recognition, ser

Key facts. Emotional speech synthesis and recognition: 259 papers on arXiv since 2021 (data through 19 Oct 2026). 30 of its 259 mapped papers (12%) cite “EmoDiff: Intensity Controllable Emotional Text-to-Speech with Soft-Label Guidance” (2022, arXiv 2211.09496). 8 new papers 2 Oct–19 Oct.

What’s new

2 Oct – 19 Oct 2026

8 new papers, 2 Oct – 19 Oct; 1 this week.

  1. This week · 1
  2. Speech LLMs That Breathe: Modeling Non-Verbal Vocalizations in Dialogue also fits: Audio language models and spoken dialogue
  3. Earlier · 7
  4. Too Emotional to Be Fake? Intra-Utterance Emotion Shifts Expose Failure Modes in Speech Deepfake Detectors
  5. E3-System: Expressive EmoData, Explicitly Controllable, Easy-to-Use TTS also fits: Vocoders and speech synthesis
  6. Steerspeech: Activation Steering For Emotion Control In Generated Speech
  7. Steering Follows Geometry, Not Labels: Emotion Directions in a Full-Duplex Speech Model
  8. Can Prosodic Style Be Inferred from Text Alone? Evidence from Unsupervised Acoustic Clusters also fits: Vocoders and speech synthesis
  9. Tracing a Sparse Emotion-Control Circuit in LLM-Based Text-to-Speech
  10. EMODE: Dynamic Para-Semantic Experts for Emotion-Aware Speech Language Modeling

Foundations

Most cited by papers here

  1. EmoDiff: Intensity Controllable Emotional Text-to-Speech with Soft-Label Guidance2022cited by 30 here
  2. MsEmoTTS: Multi-scale emotion transfer, prediction, and control for emotional speech synthesis2022cited by 22 here
  3. Speech Synthesis with Mixed Emotions2022cited by 19 here
  4. EmoVoice: LLM-based Emotional Text-To-Speech Model with Freestyle Text Prompting2025cited by 18 here
  5. Laugh Now Cry Later: Controlling Time-Varying Emotional States of Flow-Matching-Based Zero-Shot Text-to-Speech2024cited by 17 here

Lineage

The papers this direction builds on, oldest first.

Each paper on the line cites the one before.

  1. 2021Conditional Variational Autoencoder with Adversarial Learning for End-to-End Text-to-Speechcited by 27 here
  2. 2021A Survey on Neural Speech Synthesiscited by 15 here
  3. 2022Speech Synthesis with Mixed Emotionscited by 19 here
  4. 2022EmoDiff: Intensity Controllable Emotional Text-to-Speech with Soft-Label Guidancecited by 30 here
  5. 2024Laugh Now Cry Later: Controlling Time-Varying Emotional States of Flow-Matching-Based Zero-Shot Text-to-Speechcited by 17 here
  6. 2024EmoSphere++: Emotion-Controllable Zero-Shot Text-to-Speech via Emotion-Adaptive Spherical Vectorcited by 16 here
  7. 2025EmoVoice: LLM-based Emotional Text-To-Speech Model with Freestyle Text Promptingcited by 18 here
17 more it builds on, not in this chain
  1. 2021Grad-TTS: A Diffusion Probabilistic Model for Text-to-Speechcited by 20 here
  2. 2021WavLM: Large-Scale Self-Supervised Pre-Training for Full Stack Speech Processingcited by 58 here
  3. 2022MsEmoTTS: Multi-scale emotion transfer, prediction, and control for emotional speech synthesiscited by 22 here
  4. 2022DailyTalk: Spoken Dialogue Dataset for Conversational Text-to-Speechcited by 16 here
  5. 2022PromptTTS: Controllable Text-to-Speech with Text Descriptionscited by 21 here
  6. 2022Robust Speech Recognition via Large-Scale Weak Supervisioncited by 93 here
  7. 2023Neural Codec Language Models are Zero-Shot Text to Speech Synthesizerscited by 27 here
  8. 2023InstructTTS: Modelling Expressive TTS in Discrete Latent Space with Natural Language Style Promptcited by 16 here
  9. 2023EmoSpeech: Guiding FastSpeech2 Towards Emotional Text to Speechcited by 14 here
  10. 2024Seed-TTS: A Family of High-Quality Versatile Speech Generation Modelscited by 19 here
  11. 2024FunAudioLLM: Voice Understanding and Generation Foundation Models for Natural Interaction Between Humans and LLMscited by 15 here
  12. 2024CosyVoice: A Scalable Multilingual Zero-shot Text-to-speech Synthesizer based on Supervised Semantic Tokenscited by 26 here
  13. 2024Qwen2-Audio Technical Reportcited by 20 here
  14. 2024F5-TTS: A Fairytaler that Fakes Fluent and Faithful Speech with Flow Matchingcited by 21 here
  15. 2024CosyVoice 2: Scalable Streaming Speech Synthesis with Large Language Modelscited by 31 here
  16. 2025CosyVoice 3: Towards In-the-wild Speech Generation via Scaling-up and Post-trainingcited by 15 here
  17. 2025IndexTTS2: A Breakthrough in Emotionally Expressive and Duration-Controlled Auto-Regressive Zero-Shot Text-to-Speechcited by 15 here

The 24 most cited of 667.

Browse

30 of 259, ranked 6–35

  1. 6EmoSphere++: Emotion-Controllable Zero-Shot Text-to-Speech via Emotion-Adaptive Spherical Vector202416
  2. 7IndexTTS2: A Breakthrough in Emotionally Expressive and Duration-Controlled Auto-Regressive Zero-Shot Text-to-Speech202515
  3. 8EmoSpeech: Guiding FastSpeech2 Towards Emotional Text to Speech202314
  4. 9Emo-DPO: Controllable Emotional Speech Synthesis through Direct Preference Optimization202413
  5. 10Emotion Rendering for Conversational Speech Synthesis with Heterogeneous Graph-Based Context Modeling202313
  6. 11Fusing ASR Outputs in Joint Training for Speech Emotion Recognition202112
  7. 12NonverbalTTS: A Public English Corpus of Text-Aligned Nonverbal Vocalizations with Emotion Annotations for Text-to-Speech202511
  8. 13EMOVIE: A Mandarin Emotion Speech Dataset with a Simple Emotional Text-to-Speech Model202111
  9. 14Cross-speaker emotion disentangling and transfer for end-to-end speech synthesis202111
  10. 15EmoSteer-TTS: Fine-Grained and Training-Free Emotion-Controllable Text-to-Speech via Activation Steering202510
All 30
  1. 16BLSP-Emo: Towards Empathetic Large Speech-Language Models202410
  2. 17Speech Emotion Recognition with ASR Transcripts: A Comprehensive Study on Word Error Rate and Fusion Techniques202410
  3. 18iEmoTTS: Toward Robust Cross-Speaker Emotion Transfer and Control for Speech Synthesis based on Disentanglement between Prosody and Timbre202210
  4. 19An Overview of Affective Speech Synthesis and Conversion in the Deep Learning Era202210
  5. 20Enhancing Speaking Styles in Conversational Text-to-Speech Synthesis with Graph-based Multi-modal Context Modeling202110
  6. 21Steering Language Model to Stable Speech Emotion Recognition via Contextual Perception and Chain of Thought20259
  7. 22ED-TTS: Multi-Scale Emotion Modeling using Cross-Domain Emotion Diarization for Emotional Speech Synthesis20249
  8. 23EmoKnob: Enhance Voice Cloning with Fine-Grained Emotion Control20248
  9. 24Hierarchical Emotion Prediction and Control in Text-to-Speech Synthesis20247
  10. 25ZET-Speech: Zero-shot adaptive Emotion-controllable Text-to-Speech Synthesis with Diffusion and Style-based Models20237
  11. 26E-chat: Emotion-sensitive Spoken Dialogue System with Large Language Models20237
  12. 27FCTalker: Fine and Coarse Grained Context Modeling for Expressive Conversational Speech Synthesis20227
  13. 28NVSpeech: An Integrated and Scalable Pipeline for Human-Like Speech Modeling with Paralinguistic Vocalizations20256
  14. 29MNV-17: A High-Quality Performative Mandarin Dataset for Nonverbal Vocalization Recognition in Speech20256
  15. 30Making Flow-Matching-Based Zero-Shot Text-to-Speech Laugh as You Like20246
  16. 31Emotional Dimension Control in Language Model-Based Text-to-Speech: Spanning a Broad Spectrum of Human Emotions20246
  17. 32Fine-grained Emotional Control of Text-To-Speech: Learning To Rank Inter- And Intra-Class Emotion Intensities20236
  18. 33JVNV: A Corpus of Japanese Emotional Speech with Verbal Content and Nonverbal Expressions20236
  19. 34CONCSS: Contrastive-based Context Comprehension for Dialogue-appropriate Prosody in Conversational Speech Synthesis20236
  20. 35SECap: Speech Emotion Captioning with Large Language Model20236