mlsanityData to 7 Oct 2026

Seen and Unseen emotional style transfer for voice conversion with a new emotional speech dataset

arXiv 2010.14794 ↗First posted 28 Oct 2020cs.SD cs.CL eess.AScited by 119

Not on the map. Why?

Abstract (arXiv)

Emotional voice conversion aims to transform emotional prosody in speech while preserving the linguistic content and speaker identity. Prior studies show that it is possible to disentangle emotional prosody using an encoder-decoder network conditioned on discrete representation, such as one-hot emotion labels. Such networks learn to remember a fixed set of emotional styles. In this paper, we propose a novel framework based on variational auto-encoding Wasserstein generative adversarial network (VAW-GAN), which makes use of a pre-trained speech emotion recognition (SER) model to transfer emotional style during training and at run-time inference. In this way, the network is able to transfer both seen and unseen emotional style to a new utterance. We show that the proposed framework achieves remarkable performance by consistently outperforming the baseline framework. This paper also marks the release of an emotional speech dataset (ESD) for voice conversion, which has multiple speakers and languages.

Key facts. Cited by 119 papers in the map (data through 7 Oct 2026). First posted on arXiv 28 Oct 2020.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 12; 2022: 19; 2023: 17; 2024: 24; 2025: 22; 2026: 25.

119 papers.

Sort:

By area:

  1. 2024MaskGCT: Zero-Shot Text-to-Speech with Masked Generative Codec Transformercited by 159
  2. 2021Emotional Voice Conversion: Theory, Databases and ESDcited by 113
  3. 2025Spark-TTS: An Efficient LLM-Based Text-to-Speech Model with Single-Stream Decoupled Speech Tokenscited by 109
  4. 2025IndexTTS2: A Breakthrough in Emotionally Expressive and Duration-Controlled Auto-Regressive Zero-Shot Text-to-Speechcited by 85
  5. 2022FreeVC: Towards High-Quality Text-Free One-Shot Voice Conversioncited by 79
  6. 2021StarGANv2-VC: A Diverse, Unsupervised, Non-parallel Framework for Natural-Sounding Voice Conversioncited by 57
  7. 2023TextrolSpeech: A Text Style Control Speech Corpus With Codec Language Text-to-Speech Modelscited by 48
  8. 2023PromptTTS 2: Describing and Generating Voices with Text Promptcited by 47
  9. 2025DeSTA2.5-Audio: Toward General-Purpose Large Audio Language Model with Self-Generated Cross-Modal Alignmentcited by 47
  10. 2024The VoicePrivacy 2024 Challenge Evaluation Plancited by 39
  11. 2024EmoBox: Multilingual Multi-corpus Speech Emotion Recognition Toolkit and Benchmarkcited by 39
  12. 2024MEMO: Memory-Guided Diffusion for Expressive Talking Video Generationcited by 38
38 more
  1. 2022EmoDiff: Intensity Controllable Emotional Text-to-Speech with Soft-Label Guidancecited by 37
  2. 2022Emotion Intensity and its Control for Emotional Voice Conversioncited by 35
  3. 2022StyleTTS: A Style-Based Generative Model for Natural and Diverse Text-to-Speech Synthesiscited by 32
  4. 2022An Overview of Affective Speech Synthesis and Conversion in the Deep Learning Eracited by 32
  5. 2021Textless Speech Emotion Conversion using Discrete and Decomposed Representationscited by 29
  6. 2022Speech Synthesis with Mixed Emotionscited by 28
  7. 2023Make-A-Voice: Unified Voice Synthesis With Discrete Representationcited by 27
  8. 2022GenerSpeech: Towards Style Transfer for Generalizable Out-Of-Domain Text-to-Speechcited by 21
  9. 2022Controllable Data Generation by Deep Learning: A Reviewcited by 20
  10. 2021Reinforcement Learning for Emotional Text-to-Speech Synthesis with Improved Emotion Discriminabilitycited by 19
  11. 2023EmoMix: Emotion Mixing via Diffusion Models for Emotional Speech Synthesiscited by 19
  12. 2024Enhancing Zero-shot Text-to-Speech Synthesis with Human Feedbackcited by 19
  13. 2021EMOVIE: A Mandarin Emotion Speech Dataset with a Simple Emotional Text-to-Speech Modelcited by 18
  14. 2023PromptVC: Flexible Stylistic Voice Conversion in Latent Space Driven by Natural Language Promptscited by 18
  15. 2021Limited Data Emotional Voice Conversion Leveraging Text-to-Speech: Two-stage Sequence-to-Sequence Trainingcited by 17
  16. 2022Exploiting modality-invariant feature for robust multimodal emotion recognition with missing modalitiescited by 16
  17. 2021Fine-grained style control in Transformer-based Text-to-speech Synthesiscited by 15
  18. 2022A Review of Speech-centric Trustworthy Machine Learning: Privacy, Safety, and Fairnesscited by 13
  19. 2023Speech Emotion Diarization: Which Emotion Appears When?cited by 13
  20. 2025OSUM: Advancing Open Speech Understanding Models with Limited Resources in Academiacited by 13
  21. 2021Speaker Attentive Speech Emotion Recognitioncited by 9
  22. 2023Nonparallel Emotional Voice Conversion For Unseen Speaker-Emotion Pairs Using Dual Domain Adversarial Network & Virtual Domain Pairingcited by 9
  23. 2024ED-TTS: Multi-Scale Emotion Modeling using Cross-Domain Emotion Diarization for Emotional Speech Synthesiscited by 9
  24. 2022CTL-MTNet: A Novel CapsNet and Transfer Learning-Based Mixed Task Net for the Single-Corpus and Cross-Corpus Speech Emotion Recognitioncited by 8
  25. 2023QI-TTS: Questioning Intonation Control for Emotional Speech Synthesiscited by 8
  26. 2024Controlling Emotion in Text-to-Speech with Natural Language Promptscited by 8
  27. 2024Emotional Dimension Control in Language Model-Based Text-to-Speech: Spanning a Broad Spectrum of Human Emotionscited by 8
  28. 2026AudioSAE: Towards Understanding of Audio-Processing Models with Sparse AutoEncoderscited by 8
  29. 2021An Improved StarGAN for Emotional Voice Conversion: Enhancing Voice Quality and Data Augmentationcited by 7
  30. 2022An Overview of Recent Work in Media Forensics: Methods and Threatscited by 7
  31. 2022Style-Label-Free: Cross-Speaker Style Transfer by Quantized VAE and Speaker-wise Normalization in Speech Synthesiscited by 7
  32. 2023Improving Prosody for Cross-Speaker Style Transfer by Semi-Supervised Style Extractor and Hierarchical Modeling in Speech Synthesiscited by 7
  33. 2023EMNS /Imz/ Corpus: An emotive single-speaker dataset for narrative storytelling in games, television and graphic novelscited by 7
  34. 2024A Review of Human Emotion Synthesis Based on Generative Technologycited by 7
  35. 2026CoCoEmo: Composable and Controllable Human-Like Emotional TTS via Activation Steeringcited by 7
  36. 2024Style Mixture of Experts for Expressive Text-To-Speech Synthesiscited by 6
  37. 2024Rasa: Building Expressive Speech Synthesis Systems for Indian Languages in Low-resource Settingscited by 6
  38. 2025UniSS: Unified Expressive Speech-to-Speech Translation with Your Voicecited by 6

50 most cited of 119 All 119 (JSON)