# DreamVoice: Text-Guided Voice Conversion

arXiv: https://arxiv.org/abs/2406.16314 · HTML: https://mlsanity.com/paper/2406.16314/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 24 Jun 2024.

## Built on by (most cited first, 7 of 7)

- [CapSpeech: Enabling Downstream Applications in Style-Captioned Text-to-Speech](https://mlsanity.com/paper/2506.02863/) (2025)
- [FasterVoiceGrad: Faster One-step Diffusion-Based Voice Conversion with Adversarial Diffusion Conversion Distillation](https://mlsanity.com/paper/2508.17868/) (2025)
- [JAVEDIT: Joint Audio-Visual Instruction-Guided Video Editing with Agentic Data Curation](https://mlsanity.com/paper/2606.03168/) (2026)
- [VoiceDesigner: Text-to-Voice Generation and Editing via Unified Diffusion Modeling and Data Augmentation](https://mlsanity.com/paper/2608.13613/) (2026)
- [Expressive Speech Retrieval using Natural Language Descriptions of Speaking Style](https://mlsanity.com/paper/2508.11187/) (2025)
- [TargetSEC: Plug-and-Play In-the-Wild Speech Emotion Conversion via Arousal-Conditioned Latent Style Diffusion](https://mlsanity.com/paper/2606.07293/) (2026)
- [ParA-LLM: A Unified Approach to Paralinguistic and Acoustic Speech Understanding](https://mlsanity.com/paper/2609.22771/) (2026)

## Abstract (arXiv)

Generative voice technologies are rapidly evolving, offering opportunities for more personalized and inclusive experiences. Traditional one-shot voice conversion (VC) requires a target recording during inference, limiting ease of usage in generating desired voice timbres. Text-guided generation offers an intuitive solution to convert voices to desired "DreamVoices" according to the users' needs. Our paper presents two major contributions to VC technology: (1) DreamVoiceDB, a robust dataset of voice timbre annotations for 900 speakers from VCTK and LibriTTS. (2) Two text-guided VC methods: DreamVC, an end-to-end diffusion-based text-guided VC model; and DreamVG, a versatile text-to-voice generation plugin that can be combined with any one-shot VC models. The experimental results demonstrate that our proposed methods trained on the DreamVoiceDB dataset generate voice timbres accurately aligned with the text prompt and achieve high-quality VC.