# An Overview & Analysis of Sequence-to-Sequence Emotional Voice Conversion

arXiv: https://arxiv.org/abs/2203.15873 · HTML: https://mlsanity.com/paper/2203.15873/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Builds on 18 of its 38 references that are in the map.
- First posted on arXiv 29 Mar 2022.

## Builds on (most cited first, 18 of 18)

- [Neural Machine Translation by Jointly Learning to Align and Translate](https://mlsanity.com/paper/1409.0473/) (2014)
- [Sequence to Sequence Learning with Neural Networks](https://mlsanity.com/paper/1409.3215/) (2014)
- [HiFi-GAN: Generative Adversarial Networks for Efficient and High Fidelity Speech Synthesis](https://mlsanity.com/paper/2010.05646/) (2020)
- [Tacotron: Towards End-to-End Speech Synthesis](https://mlsanity.com/paper/1703.10135/) (2017)
- [Parallel WaveGAN: A fast waveform generation model based on generative adversarial networks with multi-resolution spectrogram](https://mlsanity.com/paper/1910.11480/) (2019)
- [Joint CTC-Attention based End-to-End Speech Recognition using Multi-task Learning](https://mlsanity.com/paper/1609.06773/) (2016)
- [Efficient Neural Audio Synthesis](https://mlsanity.com/paper/1802.08435/) (2018)
- [State-of-the-art Speech Recognition With Sequence-to-Sequence Models](https://mlsanity.com/paper/1712.01769/) (2017)
- [A Survey on Neural Speech Synthesis](https://mlsanity.com/paper/2106.15561/) (2021)
- [Emotional Voice Conversion: Theory, Databases and ESD](https://mlsanity.com/paper/2105.14762/) (2021)
- [The Emotional Voices Database: Towards Controlling the Emotion Dimension in Voice Generation Systems](https://mlsanity.com/paper/1806.09514/) (2018)
- [Emotion Intensity and its Control for Emotional Voice Conversion](https://mlsanity.com/paper/2201.03967/) (2022)
- [Voice Transformer Network: Sequence-to-Sequence Voice Conversion Using Transformer with Text-to-Speech Pretraining](https://mlsanity.com/paper/1912.06813/) (2019)
- [Transforming Spectrum and Prosody for Emotional Voice Conversion with Non-Parallel Training Data](https://mlsanity.com/paper/2002.00198/) (2020)
- [Converting Anyone's Emotion: Towards Speaker-Independent Emotional Voice Conversion](https://mlsanity.com/paper/2005.07025/) (2020)
- [Limited Data Emotional Voice Conversion Leveraging Text-to-Speech: Two-stage Sequence-to-Sequence Training](https://mlsanity.com/paper/2103.16809/) (2021)
- [Emotional Voice Conversion using Multitask Learning with Text-to-speech](https://mlsanity.com/paper/1911.06149/) (2019)
- [Improving Sequence-to-Sequence Acoustic Modeling by Adding Text-Supervision](https://mlsanity.com/paper/1811.08111/) (2018)

## Built on by (most cited first, 6 of 6)

- [An Overview of Affective Speech Synthesis and Conversion in the Deep Learning Era](https://mlsanity.com/paper/2210.03538/) (2022)
- [EmoSphere++: Emotion-Controllable Zero-Shot Text-to-Speech via Emotion-Adaptive Spherical Vector](https://mlsanity.com/paper/2411.02625/) (2024)
- [DurFlex-EVC: Duration-Flexible Emotional Voice Conversion Leveraging Discrete Representations without Text Alignment](https://mlsanity.com/paper/2401.08095/) (2024)
- [Audio-based Kinship Verification Using Age Domain Conversion](https://mlsanity.com/paper/2410.11120/) (2024)
- [Neuron-Level Emotion Control in Speech-Generative Large Audio-Language Models](https://mlsanity.com/paper/2603.17231/) (2026)
- [V.O.I.C.E (Voice, Ownership, Identity, Control, Expression): Risk Taxonomy of Synthetic Voice Generation From Empirical Data](https://mlsanity.com/paper/2604.24794/) (2026)

## Abstract (arXiv)

Emotional voice conversion (EVC) focuses on converting a speech utterance from a source to a target emotion; it can thus be a key enabling technology for human-computer interaction applications and beyond. However, EVC remains an unsolved research problem with several challenges. In particular, as speech rate and rhythm are two key factors of emotional conversion, models have to generate output sequences of differing length. Sequence-to-sequence modelling is recently emerging as a competitive paradigm for models that can overcome those challenges. In an attempt to stimulate further research in this promising new direction, recent sequence-to-sequence EVC papers were systematically investigated and reviewed from six perspectives: their motivation, training strategies, model architectures, datasets, model inputs, and evaluation methods. This information is organised to provide the research community with an easily digestible overview of the current state-of-the-art. Finally, we discuss existing challenges of sequence-to-sequence EVC.