# IQDUBBING: Prosody modeling based on discrete self-supervised speech representation for expressive voice conversion

arXiv: https://arxiv.org/abs/2201.00269 · HTML: https://mlsanity.com/paper/2201.00269/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 2 Jan 2022.

## Built on by (most cited first, 7 of 7)

- [EmphAssess : a Prosodic Benchmark on Assessing Emphasis Transfer in Speech-to-Speech Models](https://mlsanity.com/paper/2312.14069/) (2023)
- [Face-Driven Zero-Shot Voice Conversion with Memory-based Face-Voice Alignment](https://mlsanity.com/paper/2309.09470/) (2023)
- [DualVC: Dual-mode Voice Conversion using Intra-model Knowledge Distillation and Hybrid Predictive Coding](https://mlsanity.com/paper/2305.12425/) (2023)
- [NaturalVoices: A Large-Scale, Spontaneous and Emotional Podcast Dataset for Voice Conversion](https://mlsanity.com/paper/2511.00256/) (2025)
- [Converting Anyone's Voice: End-to-End Expressive Voice Conversion with a Conditional Diffusion Model](https://mlsanity.com/paper/2405.01730/) (2024)
- [Takin-VC: Expressive Zero-Shot Voice Conversion via Adaptive Hybrid Content Encoding and Enhanced Timbre Modeling](https://mlsanity.com/paper/2410.01350/) (2024)
- [Automatic Speech Disentanglement for Voice Conversion using Rank Module and Speech Augmentation](https://mlsanity.com/paper/2306.12259/) (2023)

## Abstract (arXiv)

Prosody modeling is important, but still challenging in expressive voice conversion. As prosody is difficult to model, and other factors, e.g., speaker, environment and content, which are entangled with prosody in speech, should be removed in prosody modeling. In this paper, we present IQDubbing to solve this problem for expressive voice conversion. To model prosody, we leverage the recent advances in discrete self-supervised speech representation (DSSR). Specifically, prosody vector is first extracted from pre-trained VQ-Wav2Vec model, where rich prosody information is embedded while most speaker and environment information are removed effectively by quantization. To further filter out the redundant information except prosody, such as content and partial speaker information, we propose two kinds of prosody filters to sample prosody from the prosody vector. Experiments show that IQDubbing is superior to baseline and comparison systems in terms of speech quality while maintaining prosody consistency and speaker similarity.