mlsanityData to 7 Oct 2026

MeanVoiceFlow2: Joint Optimization of Mean Flow and Content Encoder for Fast One-Step Zero-Shot Voice Conversion

arXiv 2609.40087 ↗First posted 30 Sep 2026cs.SD cs.AI cs.LGcited by 0

On the map: Diffusion models → Audio, speech and music

Abstract (arXiv)

Flow-matching approaches to voice conversion (VC) have gained attention owing to their high speech quality and strong speaker similarity. Among them, one-step models such as MeanVoiceFlow are particularly attractive because they enable efficient inference; however, their reliance on a computationally intensive content encoder remains a bottleneck. We therefore propose MeanVoiceFlow2, a framework that jointly optimizes a flow-based conversion module and a computationally efficient content encoder. The model is trained through conversion distillation using MeanVoiceFlow and the reconstruction of real data. We further incorporate diffusion-GAN training with sample mixing and teacher-guided conditioning augmentation to enhance realism and disentanglement. Experiments on zero-shot VC showed that MeanVoiceFlow2 achieved higher perceptual quality and approximately 9× faster inference than MeanVoiceFlow while maintaining comparable speaker similarity. Audio samples are available at https://www.kecl.ntt.co.jp/people/kaneko.takuhiro/projects/meanvoiceflow2/.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Audio, speech and music, one of 14 directions in Diffusion models. Builds on 47 of its 55 references that are in the map. First posted on arXiv 30 Sep 2026.

Builds on

47 of 55 references.

Closest ancestors · 11

Same direction, newest first

  1. 2026MeanVoiceFlow: One-step Nonparallel Voice Conversion with Mean Flows20 Feb
  2. 2025LatentVoiceGrad: Nonparallel Voice Conversion with Latent Diffusion/Flow-Matching Models10 Sep
  3. 2025FasterVoiceGrad: Faster One-step Diffusion-Based Voice Conversion with Adversarial Diffusion Conversion Distillation25 Aug
  4. 2025ReFlow-VC: Zero-shot Voice Conversion Based on Rectified Flow and Speaker Feature Optimization1 Jun
  5. 2025VoicePrompter: Robust Zero-Shot Voice Conversion with Voice Prompt and Conditional Flow Matching29 Jan
  6. 2024StableVC: Style Controllable Zero-Shot Voice Conversion with Conditional Flow Matching6 Dec
  7. 2024FastVoiceGrad: One-step Diffusion-Based Voice Conversion with Adversarial Conditional Diffusion Distillation3 Sep
  8. 2023Diff-HierVC: Diffusion-based Hierarchical Voice Conversion with Robust Pitch Generation and Masked Prior for Zero-shot Speaker Adaptation8 Nov
  9. 2023DDDM-VC: Decoupled Denoising Diffusion Models with Disentangled Representation and Prior Mixup for Verified Robust Voice Conversion25 May
  10. 2021Diffusion-Based Voice Conversion with Fast Maximum Likelihood Sampling Scheme28 Sep
  11. 2021DiffSVC: A Diffusion Probabilistic Model for Singing Voice Conversion28 May

Other references · 36

  1. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  2. 2020Denoising Diffusion Probabilistic Modelscited by 20,851
  3. 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,604
  4. 2015Deep Unsupervised Learning using Nonequilibrium Thermodynamicscited by 7,005
  5. 2022Flow Matching for Generative Modelingcited by 5,426
  6. 2017Neural Discrete Representation Learningcited by 4,650
  7. 2024Scaling Rectified Flow Transformers for High-Resolution Image Synthesiscited by 4,094
  8. 2019Generative Modeling by Estimating Gradients of the Data Distributioncited by 4,067
  9. 2022Robust Speech Recognition via Large-Scale Weak Supervisioncited by 3,972
  10. 2022Flow Straight and Fast: Learning to Generate and Transfer Data with Rectified Flowcited by 3,550
  11. 2020Conformer: Convolution-augmented Transformer for Speech Recognitioncited by 1,838
  12. 2021WavLM: Large-Scale Self-Supervised Pre-Training for Full Stack Speech Processingcited by 1,609
24 more
  1. 2016Instance Normalization: The Missing Ingredient for Fast Stylizationcited by 1,187
  2. 2020HiFi-GAN: Generative Adversarial Networks for Efficient and High Fidelity Speech Synthesiscited by 1,139
  3. 2014NICE: Non-linear Independent Components Estimationcited by 1,052
  4. 2016Least Squares Generative Adversarial Networkscited by 1,000
  5. 2022Building Normalizing Flows with Stochastic Interpolantscited by 964
  6. 2016Language Modeling with Gated Convolutional Networkscited by 688
  7. 2019LibriTTS: A Corpus Derived from LibriSpeech for Text-to-Speechcited by 639
  8. 2016Weight Normalization: A Simple Reparameterization to Accelerate Training of Deep Neural Networkscited by 541
  9. 2025Mean Flows for One-step Generative Modelingcited by 524
  10. 2022UTMOS: UTokyo-SaruLab System for VoiceMOS Challenge 2022cited by 381
  11. 2018Transfer Learning from Speaker Verification to Multispeaker Text-To-Speech Synthesiscited by 280
  12. 2022BigVGAN: A Universal Neural Vocoder with Large-Scale Trainingcited by 245
  13. 2020DNSMOS: A Non-Intrusive Perceptual Objective Speech Quality metric to evaluate Noise Suppressorscited by 235
  14. 2022Diffusion-GAN: Training GANs with Diffusioncited by 203
  15. 2024Consistency Models Made Easycited by 140
  16. 2018StarGAN-VC: Non-parallel many-to-many voice conversion with star generative adversarial networkscited by 78
  17. 2022The VoiceMOS Challenge 2022cited by 57
  18. 2020AGAIN-VC: A One-shot Voice Conversion using Activation Guidance and Adaptive Instance Normalizationcited by 44
  19. 2020Any-to-Many Voice Conversion with Location-Relative Sequence-to-Sequence Modelingcited by 42
  20. 2017Voice Conversion from Unaligned Corpora using Variational Autoencoding Wasserstein Generative Adversarial Networkscited by 38
  21. 2019Blow: a single-scale hyperconditioned flow for non-parallel raw-audio voice conversioncited by 29
  22. 2020VoiceGrad: Non-Parallel Any-to-Many Voice Conversion with Annealed Langevin Dynamicscited by 14
  23. 2025Enhancing Expressive Voice Conversion with Discrete Pitch-Conditioned Flow Matching Modelcited by 8
  24. 2025Vocoder-Projected Feature Discriminatorcited by 2

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025MeanVC: Lightweight and Streaming Zero-Shot Voice Conversion via Mean Flows
  2. 2025Rhythm Controllable and Efficient Zero-Shot Voice Conversion via Shortcut Flow Matching
  3. 2026MeanVC 2: Robust Low-Latency Streaming Zero-Shot Voice Conversion ↗
  4. 2024Zero-Shot Voice Conversion via Content-Aware Timbre Ensemble and Conditional Flow Matching
  5. 2025AdaptVC: High Quality Voice Conversion with Adaptive Learning
  6. 2024Zero-shot Voice Conversion with Diffusion Transformers
  7. 2025EZ-VC: Easy Zero-shot Any-to-Any Voice Conversion
  8. 2026One-Step Voice Conversion by Learning kNN Transport in WavLM Space ↗