# Fast-VGAN: Lightweight Voice Conversion with Explicit Control of F0 and Duration Parameters

arXiv: https://arxiv.org/abs/2507.04817 · HTML: https://mlsanity.com/paper/2507.04817/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 31 of its 49 references that are in the map.
- First posted on arXiv 7 Jul 2025.

## Builds on (most cited first, 30 of 31)

- [wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representations](https://mlsanity.com/paper/2006.11477/) (2020)
- [HuBERT: Self-Supervised Speech Representation Learning by Masked Prediction of Hidden Units](https://mlsanity.com/paper/2106.07447/) (2021)
- [Conformer: Convolution-augmented Transformer for Speech Recognition](https://mlsanity.com/paper/2005.08100/) (2020)
- [HiFi-GAN: Generative Adversarial Networks for Efficient and High Fidelity Speech Synthesis](https://mlsanity.com/paper/2010.05646/) (2020)
- [Grad-TTS: A Diffusion Probabilistic Model for Text-to-Speech](https://mlsanity.com/paper/2105.06337/) (2021)
- [AUTOVC: Zero-Shot Voice Style Transfer with Only Autoencoder Loss](https://mlsanity.com/paper/1905.05879/) (2019)
- [Seen and Unseen emotional style transfer for voice conversion with a new emotional speech dataset](https://mlsanity.com/paper/2010.14794/) (2020)
- [An Overview of Voice Conversion and its Challenges: From Statistical Modeling to Deep Learning](https://mlsanity.com/paper/2008.03648/) (2020)
- [CREPE: A Convolutional Representation for Pitch Estimation](https://mlsanity.com/paper/1802.06182/) (2018)
- [Diffusion-Based Voice Conversion with Fast Maximum Likelihood Sampling Scheme](https://mlsanity.com/paper/2109.13821/) (2021)
- [EXPRESSO: A Benchmark and Analysis of Discrete Expressive Speech Resynthesis](https://mlsanity.com/paper/2308.05725/) (2023)
- [StarGAN-VC: Non-parallel many-to-many voice conversion with star generative adversarial networks](https://mlsanity.com/paper/1806.02169/) (2018)
- [End-to-End Adversarial Text-to-Speech](https://mlsanity.com/paper/2006.03575/) (2020)
- [Robust and fine-grained prosody control of end-to-end speech synthesis](https://mlsanity.com/paper/1811.02122/) (2018)
- [DDDM-VC: Decoupled Denoising Diffusion Models with Disentangled Representation and Prior Mixup for Verified Robust Voice Conversion](https://mlsanity.com/paper/2305.15816/) (2023)
- [Revisiting Over-Smoothness in Text to Speech](https://mlsanity.com/paper/2202.13066/) (2022)
- [Parallel-Data-Free Voice Conversion Using Cycle-Consistent Adversarial Networks](https://mlsanity.com/paper/1711.11293/) (2017)
- [Speech Synthesis with Mixed Emotions](https://mlsanity.com/paper/2208.05890/) (2022)
- [AVQVC: One-shot Voice Conversion by Vector Quantization with applying contrastive learning](https://mlsanity.com/paper/2202.10020/) (2022)
- [How Far Are We from Robust Voice Conversion: A Survey](https://mlsanity.com/paper/2011.12063/) (2020)
- [Unsupervised Learning of Disentangled Speech Content and Style Representation](https://mlsanity.com/paper/2010.12973/) (2020)
- [Disentanglement of Emotional Style and Speaker Identity for Expressive Voice Conversion](https://mlsanity.com/paper/2110.10326/) (2021)
- [Emotional Prosody Control for Speech Generation](https://mlsanity.com/paper/2111.04730/) (2021)
- [NVC-Net: End-to-End Adversarial Voice Conversion](https://mlsanity.com/paper/2106.00992/) (2021)
- [Disentangling Prosody Representations with Unsupervised Speech Reconstruction](https://mlsanity.com/paper/2212.06972/) (2022)
- [ControlVC: Zero-Shot Voice Conversion with Time-Varying Controls on Pitch and Speed](https://mlsanity.com/paper/2209.11866/) (2022)
- [Learning Explicit Prosody Models and Deep Speaker Embeddings for Atypical Voice Conversion](https://mlsanity.com/paper/2011.01678/) (2020)
- [HiFi-VC: High Quality ASR-Based Voice Conversion](https://mlsanity.com/paper/2203.16937/) (2022)
- [Highly Controllable Diffusion-based Any-to-Any Voice Conversion Model with Frame-level Prosody Feature](https://mlsanity.com/paper/2309.03364/) (2023)
- [Voice Reenactment with F0 and timing constraints and adversarial learning of conversions](https://mlsanity.com/paper/2110.03744/) (2021)

## Built on by (most cited first, 2 of 2)

- [PitchFlower: A flow-based neural audio codec with pitch controllability](https://mlsanity.com/paper/2510.25566/) (2025)
- [Understanding Hyperspherical Geometry of ECAPA-TDNN Embedding and Its Impact on Zero-Shot Voice Conversion](https://mlsanity.com/paper/2609.24688/) (2026)

## Abstract (arXiv)

Precise control over speech characteristics, such as pitch, duration, and speech rate, remains a significant challenge in the field of voice conversion. The ability to manipulate parameters like pitch and syllable rate is an important element for effective identity conversion, but can also be used independently for voice transformation, achieving goals that were historically addressed by vocoder-based methods. In this work, we explore a convolutional neural network-based approach that aims to provide means for modifying fundamental frequency (F0), phoneme sequences, intensity, and speaker identity. Rather than relying on disentanglement techniques, our model is explicitly conditioned on these factors to generate mel spectrograms, which are then converted into waveforms using a universal neural vocoder. Accordingly, during inference, F0 contours, phoneme sequences, and speaker embeddings can be freely adjusted, allowing for intuitively controlled voice transformations. We evaluate our approach on speaker conversion and expressive speech tasks using both perceptual and objective metrics. The results suggest that the proposed method offers substantial flexibility, while maintaining high intelligibility and speaker similarity.