# AttS2S-VC: Sequence-to-Sequence Voice Conversion with Attention and Context Preservation Mechanisms

arXiv: https://arxiv.org/abs/1811.04076 · HTML: https://mlsanity.com/paper/1811.04076/

## Key facts

- Cited by 22 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 9 Nov 2018.

## Built on by (most cited first, 22 of 22)

- [Emotional Voice Conversion: Theory, Databases and ESD](https://mlsanity.com/paper/2105.14762/) (2021)
- [Deepfakes Generation and Detection: State-of-the-art, open challenges, countermeasures, and way forward](https://mlsanity.com/paper/2103.00484/) (2021)
- [StarGANv2-VC: A Diverse, Unsupervised, Non-parallel Framework for Natural-Sounding Voice Conversion](https://mlsanity.com/paper/2107.10394/) (2021)
- [Emotion Intensity and its Control for Emotional Voice Conversion](https://mlsanity.com/paper/2201.03967/) (2022)
- [An Overview of Affective Speech Synthesis and Conversion in the Deep Learning Era](https://mlsanity.com/paper/2210.03538/) (2022)
- [MaskCycleGAN-VC: Learning Non-parallel Voice Conversion with Filling in Frames](https://mlsanity.com/paper/2102.12841/) (2021)
- [Limited Data Emotional Voice Conversion Leveraging Text-to-Speech: Two-stage Sequence-to-Sequence Training](https://mlsanity.com/paper/2103.16809/) (2021)
- [Adversarially learning disentangled speech representations for robust multi-factor voice conversion](https://mlsanity.com/paper/2102.00184/) (2021)
- [DurFlex-EVC: Duration-Flexible Emotional Voice Conversion Leveraging Discrete Representations without Text Alignment](https://mlsanity.com/paper/2401.08095/) (2024)
- [Non-autoregressive sequence-to-sequence voice conversion](https://mlsanity.com/paper/2104.06793/) (2021)
- [A Preliminary Study of a Two-Stage Paradigm for Preserving Speaker Identity in Dysarthric Voice Conversion](https://mlsanity.com/paper/2106.01415/) (2021)
- [Rhythm Modeling for Voice Conversion](https://mlsanity.com/paper/2307.06040/) (2023)
- [The Sound of Silence: Efficiency of First Digit Features in Synthetic Audio Detection](https://mlsanity.com/paper/2210.02746/) (2022)
- [RW-Resnet: A Novel Speech Anti-Spoofing Model Using Raw Waveform](https://mlsanity.com/paper/2108.05684/) (2021)
- [All-for-One and One-For-All: Deep learning-based feature fusion for Synthetic Speech Detection](https://mlsanity.com/paper/2307.15555/) (2023)
- [Axial Residual Networks for CycleGAN-based Voice Conversion](https://mlsanity.com/paper/2102.08075/) (2021)
- [Conditional Deep Hierarchical Variational Autoencoder for Voice Conversion](https://mlsanity.com/paper/2112.02796/) (2021)
- [Enhancing Zero-Shot Many to Many Voice Conversion with Self-Attention VAE](https://mlsanity.com/paper/2203.16037/) (2022)
- [ConvNeXt Based Neural Network for Audio Anti-Spoofing](https://mlsanity.com/paper/2209.06434/) (2022)
- [MoEVC: A Mixture-of-experts Voice Conversion System with Sparse Gating Mechanism for Accelerating Online Computation](https://mlsanity.com/paper/1912.11984/) (2019)
- [Beyond Voice Identity Conversion: Manipulating Voice Attributes by Adversarial Learning of Structured Disentangled Representations](https://mlsanity.com/paper/2107.12346/) (2021)
- [Parallel and Limited Data Voice Conversion Using Stochastic Variational Deep Kernel Learning](https://mlsanity.com/paper/2309.04420/) (2023)

## Abstract (arXiv)

This paper describes a method based on a sequence-to-sequence learning (Seq2Seq) with attention and context preservation mechanism for voice conversion (VC) tasks. Seq2Seq has been outstanding at numerous tasks involving sequence modeling such as speech synthesis and recognition, machine translation, and image captioning. In contrast to current VC techniques, our method 1) stabilizes and accelerates the training procedure by considering guided attention and proposed context preservation losses, 2) allows not only spectral envelopes but also fundamental frequency contours and durations of speech to be converted, 3) requires no context information such as phoneme labels, and 4) requires no time-aligned source and target speech data in advance. In our experiment, the proposed VC framework can be trained in only one day, using only one GPU of an NVIDIA Tesla K80, while the quality of the synthesized speech is higher than that of speech converted by Gaussian mixture model-based VC and is comparable to that of speech generated by recurrent neural network-based text-to-speech synthesis, which can be regarded as an upper limit on VC performance.