mlsanityData to 7 Oct 2026

Any-to-One Sequence-to-Sequence Voice Conversion using Self-Supervised Discrete Speech Representations

arXiv 2010.12231 ↗First posted 23 Oct 2020eess.AS cs.CL cs.SDcited by 18

Not on the map. Why?

Abstract (arXiv)

We present a novel approach to any-to-one (A2O) voice conversion (VC) in a sequence-to-sequence (seq2seq) framework. A2O VC aims to convert any speaker, including those unseen during training, to a fixed target speaker. We utilize vq-wav2vec (VQW2V), a discretized self-supervised speech representation that was learned from massive unlabeled data, which is assumed to be speaker-independent and well corresponds to underlying linguistic contents. Given a training dataset of the target speaker, we extract VQW2V and acoustic features to estimate a seq2seq mapping function from the former to the latter. With the help of a pretraining method and a newly designed postprocessing technique, our model can be generalized to only 5 min of data, even outperforming the same model trained with parallel data.

Key facts. Cited by 18 papers in the map (data through 7 Oct 2026). First posted on arXiv 23 Oct 2020.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 3; 2022: 4; 2023: 5; 2024: 1; 2025: 2; 2026: 3.

18 papers.

Sort:

By area:

  1. 2022SUPERB-SG: Enhanced Speech processing Universal PERformance Benchmark for Semantic and Generative Capabilitiescited by 76
  2. 2023DDDM-VC: Decoupled Denoising Diffusion Models with Disentangled Representation and Prior Mixup for Verified Robust Voice Conversioncited by 38
  3. 2021S3PRL-VC: Open-source Voice Conversion Framework with Self-supervised Speech Representationscited by 22
  4. 2023SEF-VC: Speaker Embedding Free Zero-Shot Voice Conversion with Cross Attentioncited by 19
  5. 2022Speaking Style Conversion in the Waveform Domain Using Discrete Self-Supervised Unitscited by 15
  6. 2022A Comparative Study of Self-supervised Speech Representation Based Voice Conversioncited by 9
  7. 2023Towards General-Purpose Text-Instruction-Guided Voice Conversioncited by 6
  8. 2023High-Quality Automatic Voice Over with Accurate Alignment: Supervision through Self-Supervised Discrete Speech Unitscited by 4
  9. 2021On Prosody Modeling for ASR+TTS based Voice Conversioncited by 3
  10. 2023Reimagining Speech: A Scoping Review of Deep Learning-Powered Voice Conversioncited by 3
  11. 2025NaturalVoices: A Large-Scale, Spontaneous and Emotional Podcast Dataset for Voice Conversioncited by 2
  12. 2026CosyAccent: Duration-Controllable Accent Normalization Using Source-Synthesis Training Datacited by 2
6 more
  1. 2024Converting Anyone's Voice: End-to-End Expressive Voice Conversion with a Conditional Diffusion Modelcited by 1
  2. 2021CycleFlow: Purify Information Factors by Cycle Losscited by 0
  3. 2022DisC-VC: Disentangled and F0-Controllable Neural Voice Conversioncited by 0
  4. 2025USM-VC: Mitigating Timbre Leakage with Universal Semantic Mapping Residual Block for Voice Conversioncited by 0
  5. 2026Controllable Accent Normalization via Discrete Diffusioncited by 0
  6. 2026TokAN: Accent Normalization Using Self-Supervised Speech Tokenscited by 0