mlsanityData to 7 Oct 2026

S3PRL-VC: Open-source Voice Conversion Framework with Self-supervised Speech Representations

arXiv 2110.06280 ↗First posted 12 Oct 2021cs.SD cs.CL cs.LGcited by 22

Not on the map. Why?

Abstract (arXiv)

This paper introduces S3PRL-VC, an open-source voice conversion (VC) framework based on the S3PRL toolkit. In the context of recognition-synthesis VC, self-supervised speech representation (S3R) is valuable in its potential to replace the expensive supervised representation adopted by state-of-the-art VC systems. Moreover, we claim that VC is a good probing task for S3R analysis. In this work, we provide a series of in-depth analyses by benchmarking on the two tasks in VCC2020, namely intra-/cross-lingual any-to-one (A2O) VC, as well as an any-to-any (A2A) setting. We also provide comparisons between not only different S3Rs but also top systems in VCC2020 with supervised representations. Systematic objective and subjective evaluation were conducted, and we show that S3R is comparable with VCC2020 top systems in the A2O setting in terms of similarity, and achieves state-of-the-art in S3R-based A2A VC. We believe the extensive analysis, as well as the toolkit itself, contribute to not only the S3R community but also the VC community. The codebase is now open-sourced.

Key facts. Cited by 22 papers in the map (data through 7 Oct 2026). Builds on 18 of its 30 references that are in the map. First posted on arXiv 12 Oct 2021.

Builds on

18 of 30 references.

References · 18

  1. 2017Attention Is All You Needcited by 59,932
  2. 2020wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representationscited by 3,853
  3. 2020HiFi-GAN: Generative Adversarial Networks for Efficient and High Fidelity Speech Synthesiscited by 1,139
  4. 2017Natural TTS Synthesis by Conditioning WaveNet on Mel Spectrogram Predictionscited by 800
  5. 2021SUPERB: Speech processing Universal PERformance Benchmarkcited by 652
  6. 2017Tacotron: Towards End-to-End Speech Synthesiscited by 487
  7. 2018Transfer Learning from Speaker Verification to Multispeaker Text-To-Speech Synthesiscited by 280
  8. 2021Speech Resynthesis from Discrete Disentangled Self-Supervised Representationscited by 243
  9. 2020Exploring wav2vec 2.0 on speaker verification and language identificationcited by 93
  10. 2020Voice Conversion Challenge 2020: Intra-lingual semi-parallel and cross-lingual voice conversioncited by 75
  11. 2020FragmentVC: Any-to-Any Voice Conversion by End-to-End Extracting and Fusing Fine-Grained Voice Fragments With Attentioncited by 37
  12. 2020An Open source Implementation of ITU-T Recommendation P.808 with Validationcited by 27
6 more
  1. 2020Predictions of Subjective Ratings and Spoofing Assessments of Voice Conversion Challenge 2020 Submissionscited by 19
  2. 2020Any-to-One Sequence-to-Sequence Voice Conversion using Self-Supervised Discrete Speech Representationscited by 18
  3. 2021S2VC: A Framework for Any-to-Any Voice Conversion with Self-Supervised Pretrained Representationscited by 17
  4. 2020The Sequence-to-Sequence Baseline for the Voice Conversion Challenge 2020: Cascading ASR and TTScited by 12
  5. 2020Voice Conversion by Cascading Automatic Speech Recognition and Text-to-Speech Synthesis with Prosody Transfercited by 10
  6. 2021On Prosody Modeling for ASR+TTS based Voice Conversioncited by 3

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 8; 2023: 8; 2024: 4; 2025: 1; 2026: 1.

22 papers.

Sort:

By area:

  1. 2022FreeVC: Towards High-Quality Text-Free One-Shot Voice Conversioncited by 79
  2. 2022SUPERB-SG: Enhanced Speech processing Universal PERformance Benchmark for Semantic and Generative Capabilitiescited by 76
  3. 2023DDDM-VC: Decoupled Denoising Diffusion Models with Disentangled Representation and Prior Mixup for Verified Robust Voice Conversioncited by 38
  4. 2024A Large-Scale Evaluation of Speech Foundation Modelscited by 38
  5. 2023From Discrete Tokens to High-Fidelity Audio Using Multi-Band Diffusioncited by 22
  6. 2024Improving Robustness of LLM-based Speech Synthesis by Learning Monotonic Alignmentcited by 21
  7. 2022Speaking Style Conversion in the Waveform Domain Using Discrete Self-Supervised Unitscited by 15
  8. 2023ACE-VC: Adaptive and Controllable Voice Conversion using Explicitly Disentangled Self-supervised Speech Representationscited by 15
  9. 2023Efficient Domain Adaptation for Speech Foundation Modelscited by 13
  10. 2022Disentangled Feature Learning for Real-Time Neural Speech Codingcited by 10
  11. 2022A Comparative Study of Self-supervised Speech Representation Based Voice Conversioncited by 9
  12. 2024Self-supervised ASR Models and Features For Dysarthric and Elderly Speech Recognitioncited by 9
10 more
  1. 2025GenVC: Self-Supervised Zero-Shot Voice Conversioncited by 8
  2. 2023Towards General-Purpose Text-Instruction-Guided Voice Conversioncited by 6
  3. 2024vec2wav 2.0: Advancing Voice Conversion via Discrete Token Vocoderscited by 6
  4. 2023SelfVC: Voice Conversion With Iterative Refinement using Self Transformationscited by 5
  5. 2023Evaluating Methods for Ground-Truth-Free Foreign Accent Conversioncited by 4
  6. 2022Cross-lingual Text-To-Speech with Flow-based Voice Conversion for Improved Pronunciationcited by 3
  7. 2022A unified one-shot prosody and speaker conversion system with self-supervised discrete speech unitscited by 3
  8. 2022Self-Supervised Speech Representations Preserve Speech Characteristics while Anonymizing Voicescited by 1
  9. 2023Enhancing Speech-to-Speech Translation with Multiple TTS Targetscited by 0
  10. 2026From A to B to A: Palindromic Zero-Shot Voice Conversion with Non-Parallel Datacited by 0