# VoiceMe: Personalized voice generation in TTS

arXiv: https://arxiv.org/abs/2203.15379 · HTML: https://mlsanity.com/paper/2203.15379/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 29 Mar 2022.

## Built on by (most cited first, 6 of 6)

- [Getting aligned on representational alignment](https://mlsanity.com/paper/2310.13018/) (2023)
- [DarkStream: real-time speech anonymization with low latency](https://mlsanity.com/paper/2509.04667/) (2025)
- [Characterizing Similarities and Divergences in Conversational Tones in Humans and LLMs by Sampling with People](https://mlsanity.com/paper/2406.04278/) (2024)
- [IMaSC -- ICFOSS Malayalam Speech Corpus](https://mlsanity.com/paper/2211.12796/) (2022)
- [Controllable Generation of Artificial Speaker Embeddings through Discovery of Principal Directions](https://mlsanity.com/paper/2310.17502/) (2023)
- [Probing Perceptual Priors of MLLMs via Gibbs Sampling with Interpretable Generative Controls](https://mlsanity.com/paper/2608.27727/) (2026)

## Abstract (arXiv)

Novel text-to-speech systems can generate entirely new voices that were not seen during training. However, it remains a difficult task to efficiently create personalized voices from a high-dimensional speaker space. In this work, we use speaker embeddings from a state-of-the-art speaker verification model (SpeakerNet) trained on thousands of speakers to condition a TTS model. We employ a human sampling paradigm to explore this speaker latent space. We show that users can create voices that fit well to photos of faces, art portraits, and cartoons. We recruit online participants to collectively manipulate the voice of a speaking face. We show that (1) a separate group of human raters confirms that the created voices match the faces, (2) speaker gender apparent from the face is well-recovered in the voice, and (3) people are consistently moving towards the real voice prototype for the given face. Our results demonstrate that this technology can be applied in a wide number of applications including character voice development in audiobooks and games, personalized speech assistants, and individual voices for people with speech impairment.