mlsanityData to 14 Oct 2026

Non-autoregressive sequence-to-sequence voice conversion

arXiv 2104.06793 ↗First posted 14 Apr 2021cs.SD cs.CL eess.AScited by 7

Not on the map. Why?

Abstract (arXiv)

This paper proposes a novel voice conversion (VC) method based on non-autoregressive sequence-to-sequence (NAR-S2S) models. Inspired by the great success of NAR-S2S models such as FastSpeech in text-to-speech (TTS), we extend the FastSpeech2 model for the VC problem. We introduce the convolution-augmented Transformer (Conformer) instead of the Transformer, making it possible to capture both local and global context information from the input sequence. Furthermore, we extend variance predictors to variance converters to explicitly convert the source speaker's prosody components such as pitch and energy into the target speaker. The experimental evaluation with the Japanese speaker dataset, which consists of male and female speakers of 1,000 utterances, demonstrates that the proposed model enables us to perform more stable, faster, and better conversion than autoregressive S2S (AR-S2S) models such as Tacotron2 and Transformer.

Key facts. Cited by 7 papers in the map (data through 14 Oct 2026). Builds on 18 of its 33 references that are in the map. First posted on arXiv 14 Apr 2021.

Builds on

18 of 33 references.

References · 18

  1. 2017Attention Is All You Needcited by 60,361
  2. 2014Neural Machine Translation by Jointly Learning to Align and Translatecited by 4,548
  3. 2020Conformer: Convolution-augmented Transformer for Speech Recognitioncited by 1,863
  4. 2019Transformer-XL: Attentive Language Models Beyond a Fixed-Length Contextcited by 1,850
  5. 2017Natural TTS Synthesis by Conditioning WaveNet on Mel Spectrogram Predictionscited by 804
  6. 2020FastSpeech 2: Fast and High-Quality End-to-End Text to Speechcited by 717
  7. 2016Language Modeling with Gated Convolutional Networkscited by 702
  8. 2018ESPnet: End-to-End Speech Processing Toolkitcited by 558
  9. 2017Tacotron: Towards End-to-End Speech Synthesiscited by 490
  10. 2019Parallel WaveGAN: A fast waveform generation model based on generative adversarial networks with multi-resolution spectrogramcited by 340
  11. 2020FastPitch: Parallel Text-to-speech with Pitch Predictioncited by 151
  12. 2019ESPnet-TTS: Unified, Reproducible, and Integratable Open Source End-to-End Text-to-Speech Toolkitcited by 60
6 more
  1. 2019Voice Transformer Network: Sequence-to-Sequence Voice Conversion Using Transformer with Text-to-Speech Pretrainingcited by 29
  2. 2018AttS2S-VC: Sequence-to-Sequence Voice Conversion with Attention and Context Preservation Mechanismscited by 22
  3. 2018Improving Sequence-to-Sequence Acoustic Modeling by Adding Text-Supervisioncited by 10
  4. 2019Non-Autoregressive Neural Text-to-Speechcited by 9
  5. 2018ConvS2S-VC: Fully convolutional sequence-to-sequence voice conversioncited by 7
  6. 2020Many-to-Many Voice Transformer Networkcited by 6

Built on by

Cited by year

Citing papers per year: 2022: 3; 2023: 1; 2024: 2; 2025: 1.

7 papers.

  1. 2022A Survey on Non-Autoregressive Generation for Neural Machine Translation and Beyondcited by 64
  2. 2022Emotion Intensity and its Control for Emotional Voice Conversioncited by 35
  3. 2024DurFlex-EVC: Duration-Flexible Emotional Voice Conversion Leveraging Discrete Representations without Text Alignmentcited by 8
  4. 2023Evaluating Methods for Ground-Truth-Free Foreign Accent Conversioncited by 4
  5. 2025Conan: A Chunkwise Online Network for Zero-Shot Adaptive Voice Conversioncited by 4
  6. 2024A Pilot Study of Applying Sequence-to-Sequence Voice Conversion to Evaluate the Intelligibility of L2 Speech Using a Native Speaker's Shadowingscited by 1
  7. 2022MetaSpeech: Speech Effects Switch Along with Environment for Metaversecited by 0