# PromptSpeaker: Speaker Generation Based on Text Descriptions

arXiv: https://arxiv.org/abs/2310.05001 · HTML: https://mlsanity.com/paper/2310.05001/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 8 Oct 2023.

## Built on by (most cited first, 6 of 6)

- [LibriTTS-P: A Corpus with Speaking Style and Speaker Identity Prompts for Text-to-Speech and Style Captioning](https://mlsanity.com/paper/2406.07969/) (2024)
- [FaceSpeak: Expressive and High-Quality Speech Synthesis from Human Portraits of Different Styles](https://mlsanity.com/paper/2501.03181/) (2025)
- [HiStyle: Hierarchical Style Embedding Predictor for Text-Prompt-Guided Controllable Speech Synthesis](https://mlsanity.com/paper/2509.25842/) (2025)
- [S2Cap: A Benchmark and a Baseline for Singing Style Captioning](https://mlsanity.com/paper/2409.09866/) (2024)
- [Emputation: Identification-Guided Neural Imputation Framework](https://mlsanity.com/paper/2607.05279/) (2026)
- [VoTSpeech: Decoupling Voice Design from Speech Generation in Instruction-Following TTS](https://mlsanity.com/paper/2610.14694/) (2026)

## Abstract (arXiv)

Recently, text-guided content generation has received extensive attention. In this work, we explore the possibility of text description-based speaker generation, i.e., using text prompts to control the speaker generation process. Specifically, we propose PromptSpeaker, a text-guided speaker generation system. PromptSpeaker consists of a prompt encoder, a zero-shot VITS, and a Glow model, where the prompt encoder predicts a prior distribution based on the text description and samples from this distribution to obtain a semantic representation. The Glow model subsequently converts the semantic representation into a speaker representation, and the zero-shot VITS finally synthesizes the speaker's voice based on the speaker representation. We verify that PromptSpeaker can generate speakers new from the training set by objective metrics, and the synthetic speaker voice has reasonable subjective matching quality with the speaker prompt.