mlsanityData to 14 Oct 2026

MaskVCT: Masked Voice Codec Transformer for Zero-Shot Voice Conversion With Increased Controllability via Multiple Guidances

arXiv 2509.17143 ↗First posted 21 Sep 2025eess.AS cs.AIcited by 1

Not on the map. Why?

Abstract (arXiv)

We introduce MaskVCT, a zero-shot voice conversion (VC) model that offers multi-factor controllability through multiple classifier-free guidances (CFGs). While previous VC models rely on a fixed conditioning scheme, MaskVCT integrates diverse conditions in a single model. To further enhance robustness and control, the model can leverage continuous or quantized linguistic features to enhance intelligibility and speaker similarity, and can use or omit pitch contour to control prosody. These choices allow users to seamlessly balance speaker identity, linguistic content, and prosodic factors in a zero-shot VC setting. Extensive experiments demonstrate that MaskVCT achieves the best target speaker and accent similarities while obtaining competitive word and character error rates compared to existing baselines. Audio samples are available at https://maskvct.github.io/.

Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Builds on 37 of its 43 references that are in the map. First posted on arXiv 21 Sep 2025.

Builds on

37 of 43 references.

References · 37

  1. 2017Attention Is All You Needcited by 60,361
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  3. 2022Classifier-Free Diffusion Guidancecited by 5,847
  4. 2022Robust Speech Recognition via Large-Scale Weak Supervisioncited by 4,057
  5. 2016Categorical Reparameterization with Gumbel-Softmaxcited by 2,739
  6. 2021WavLM: Large-Scale Self-Supervised Pre-Training for Full Stack Speech Processingcited by 1,657
  7. 2019SpecAugment: A Simple Data Augmentation Method for Automatic Speech Recognitioncited by 1,380
  8. 2022MaskGIT: Masked Generative Image Transformercited by 1,099
  9. 2023Neural Codec Language Models are Zero-Shot Text to Speech Synthesizerscited by 752
  10. 2022High Fidelity Neural Audio Compressioncited by 687
  11. 2023Simple and Controllable Music Generationcited by 488
  12. 2020MLS: A Large-Scale Multilingual Dataset for Speech Researchcited by 476
25 more
  1. 2019Libri-Light: A Benchmark for ASR with Limited or No Supervisioncited by 467
  2. 2023High-Fidelity Audio Compression with Improved RVQGANcited by 431
  3. 2021Speech Resynthesis from Discrete Disentangled Self-Supervised Representationscited by 243
  4. 2024NaturalSpeech 3: Zero-Shot Speech Synthesis with Factorized Codec and Diffusion Modelscited by 226
  5. 2024MaskGCT: Zero-Shot Text-to-Speech with Masked Generative Codec Transformercited by 161
  6. 2023SoundStorm: Efficient Parallel Audio Generationcited by 114
  7. 2023LibriTTS-R: A Restored Multi-Speaker Text-to-Speech Corpuscited by 101
  8. 2023Libriheavy: a 50,000 hours ASR corpus with punctuation casing and contextcited by 85
  9. 2024The T05 System for The VoiceMOS Challenge 2024: Transfer Learning from Deep Image Classifier to Naturalness MOS Prediction of High-Quality Synthetic Speechcited by 83
  10. 2021Neural Analysis and Synthesis: Reconstructing Speech from Self-Supervised Representationscited by 81
  11. 2022FreeVC: Towards High-Quality Text-Free One-Shot Voice Conversioncited by 80
  12. 2024Self-Supervised Speech Representations are More Phonetic than Semanticcited by 71
  13. 2021Hi-Fi Multi-Speaker English TTS Datasetcited by 60
  14. 2022NANSY++: Unified Voice Synthesis with Neural Analysis and Synthesiscited by 38
  15. 2025Vevo: Controllable Zero-Shot Voice Imitation with Self-Supervised Disentanglementcited by 35
  16. 2023Diff-HierVC: Diffusion-based Hierarchical Voice Conversion with Robust Pitch Generation and Masked Prior for Zero-shot Speaker Adaptationcited by 30
  17. 2023Amphion: An Open-Source Audio, Music and Speech Generation Toolkitcited by 27
  18. 2024SyllableLM: Learning Coarse Semantic Units for Speech Language Modelscited by 24
  19. 2023VoiceLDM: Text-to-Speech with Environmental Contextcited by 23
  20. 2023CommonAccent: Exploring Large Acoustic Pretrained Models for Accent Classification Based on Common Voicecited by 20
  21. 2024Sylber: Syllabic Embedding Representation of Speech from Raw Audiocited by 19
  22. 2024DualSpeech: Enhancing Speaker-Fidelity and Text-Intelligibility Through Dual Classifier-Free Guidancecited by 11
  23. 2022PhaseAug: A Differentiable Augmentation for Speech Synthesis to Simulate One-to-Many Mappingcited by 8
  24. 2025GenVC: Self-Supervised Zero-Shot Voice Conversioncited by 8
  25. 2024Analyzing and Mitigating Inconsistency in Discrete Audio Tokens for Neural Codec Language Modelscited by 3

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 1.

1 paper.

  1. 2026Reconstruct! Don't Encode: Self-Supervised Representation Reconstruction Loss for High-Intelligibility and Low-Latency Streaming Neural Audio Codeccited by 2

Similar, not cited

Close in wording, no citation either way

  1. 2024CoDiff-VC: A Codec-Assisted Diffusion Model for Zero-shot Voice Conversion
  2. 2025Discl-VC: Disentangled Discrete Tokens and In-Context Learning for Controllable Zero-Shot Voice Conversion
  3. 2025EZ-VC: Easy Zero-shot Any-to-Any Voice Conversion
  4. 2021NoiseVC: Towards High Quality Zero-Shot Voice Conversion
  5. 2025VoicePrompter: Robust Zero-Shot Voice Conversion with Voice Prompt and Conditional Flow Matching
  6. 2025Rhythm Controllable and Efficient Zero-Shot Voice Conversion via Shortcut Flow Matching
  7. 2024StableVC: Style Controllable Zero-Shot Voice Conversion with Conditional Flow Matching
  8. 2024Takin-VC: Expressive Zero-Shot Voice Conversion via Adaptive Hybrid Content Encoding and Enhanced Timbre Modeling