P2VA: Converting Persona Descriptions into Voice Attributes for Fair and Controllable Text-to-Speech
Not on the map. Why?
Abstract (arXiv)
While persona-driven large language models (LLMs) and prompt-based text-to-speech (TTS) systems have advanced significantly, a usability gap arises when users attempt to generate voices matching their desired personas from implicit descriptions. Most users lack specialized knowledge to specify detailed voice attributes, which often leads TTS systems to misinterpret their expectations. To address these gaps, we introduce Persona-to-Voice-Attribute (P2VA), the first framework enabling voice generation automatically from persona descriptions. Our approach employs two strategies: P2VA-C for structured voice attributes, and P2VA-O for richer style descriptions. Evaluation shows our P2VA-C reduces WER by 5% and improves MOS by 0.33 points. To the best of our knowledge, P2VA is the first framework to establish a connection between persona and voice synthesis. In addition, we discover that current LLMs embed societal biases in voice attributes during the conversion process. Our experiments and findings further provide insights into the challenges of building persona-voice systems.
Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Builds on 15 of its 18 references that are in the map. First posted on arXiv 21 May 2025.
Builds on
15 of 18 references.
References · 15
- 2023Generative Agents: Interactive Simulacra of Human Behaviorcited by 3,568
- 2023MetaGPT: Meta Programming for A Multi-Agent Collaborative Frameworkcited by 1,808
- 2018Personalizing Dialogue Agents: I have a dog, do you have pets too?cited by 828
- 2024Scaling Synthetic Data Creation with 1,000,000,000 Personascited by 417
- 2024Two Tales of Persona in LLMs: A Survey of Role-Playing and Personalizationcited by 208
- 2022"I'm sorry to hear that": Finding New Biases in Language Models with a Holistic Descriptor Datasetcited by 162
- 2022PromptTTS: Controllable Text-to-Speech with Text Descriptionscited by 110
- 2023InstructTTS: Modelling Expressive TTS in Discrete Latent Space with Natural Language Style Promptcited by 92
- 2024Natural language guidance of high-fidelity text-to-speech with synthetic annotationscited by 67
- 2024Measuring Implicit Bias in Explicitly Unbiased Large Language Modelscited by 51
- 2024LLMs + Persona-Plug = Personalized LLMscited by 32
- 2023PromptTTS++: Controlling Speaker Identity in Prompt-Based Text-to-Speech Using Natural Language Descriptionscited by 25
3 more
- 2024Bias and Volatility: A Statistical Framework for Evaluating Large Language Model's Stereotypes and the Associated Generation Inconsistencycited by 15
- 2024MBIAS: Mitigating Bias in Large Language Models While Retaining Contextcited by 15
- 2023MM-TTS: Multi-modal Prompt based Style Transfer for Expressive Text-to-Speech Synthesiscited by 12
Built on by
Citing papers per year: 2026: 1.
1 paper.