# Non-parallel Emotion Conversion using a Deep-Generative Hybrid Network and an Adversarial Pair Discriminator

arXiv: https://arxiv.org/abs/2007.12932 · HTML: https://mlsanity.com/paper/2007.12932/

## Key facts

- Cited by 12 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 25 Jul 2020.

## Built on by (most cited first, 12 of 12)

- [Emotional Voice Conversion: Theory, Databases and ESD](https://mlsanity.com/paper/2105.14762/) (2021)
- [Emotion Intensity and its Control for Emotional Voice Conversion](https://mlsanity.com/paper/2201.03967/) (2022)
- [The Ethical Implications of Generative Audio Models: A Systematic Literature Review](https://mlsanity.com/paper/2307.05527/) (2023)
- [Limited Data Emotional Voice Conversion Leveraging Text-to-Speech: Two-stage Sequence-to-Sequence Training](https://mlsanity.com/paper/2103.16809/) (2021)
- [Nonparallel Emotional Voice Conversion For Unseen Speaker-Emotion Pairs Using Dual Domain Adversarial Network & Virtual Domain Pairing](https://mlsanity.com/paper/2302.10536/) (2023)
- [An Improved StarGAN for Emotional Voice Conversion: Enhancing Voice Quality and Data Augmentation](https://mlsanity.com/paper/2107.08361/) (2021)
- [An Adaptive Learning based Generative Adversarial Network for One-To-One Voice Conversion](https://mlsanity.com/paper/2104.12159/) (2021)
- [Emotion Selectable End-to-End Text-based Speech Editing](https://mlsanity.com/paper/2212.10191/) (2022)
- [Attention-based Interactive Disentangling Network for Instance-level Emotional Voice Conversion](https://mlsanity.com/paper/2312.17508/) (2023)
- [ClapFM-EVC: High-Fidelity and Flexible Emotional Voice Conversion with Dual Control from Natural Language and Speech](https://mlsanity.com/paper/2505.13805/) (2025)
- [A Deep-Bayesian Framework for Adaptive Speech Duration Modification](https://mlsanity.com/paper/2107.04973/) (2021)
- [Re-ENACT: Reinforcement Learning for Emotional Speech Generation using Actor-Critic Strategy](https://mlsanity.com/paper/2408.01892/) (2024)

## Abstract (arXiv)

We introduce a novel method for emotion conversion in speech that does not require parallel training data. Our approach loosely relies on a cycle-GAN schema to minimize the reconstruction error from converting back and forth between emotion pairs. However, unlike the conventional cycle-GAN, our discriminator classifies whether a pair of input real and generated samples corresponds to the desired emotion conversion (e.g., A to B) or to its inverse (B to A). We will show that this setup, which we refer to as a variational cycle-GAN (VC-GAN), is equivalent to minimizing the empirical KL divergence between the source features and their cyclic counterpart. In addition, our generator combines a trainable deep network with a fixed generative block to implement a smooth and invertible transformation on the input features, in our case, the fundamental frequency (F0) contour. This hybrid architecture regularizes our adversarial training procedure. We use crowd sourcing to evaluate both the emotional saliency and the quality of synthesized speech. Finally, we show that our model generalizes to new speakers by modifying speech produced by Wavenet.