Voice "Cloning" is Style Transfer
Not on the map. Why?
Abstract (arXiv)
Artificially generated speech is increasingly embedded in everyday life. Voice cloning in particular enables applications where identity preservation is important, such as completing a recording, dubbing in a new language, or preserving the voices of individuals with speech loss. However, in our work, we find that despite the term, voice cloning does not faithfully ''clone'' an individual's voice. Instead, we find that widely-used voice cloning models systematically apply style transfer to source voices. As rated by human annotators, cloned voices are perceived as more authoritative, warm, customer-service-like, and human-like compared to their sources. Human annotators also report greater trust in cloned voices than source voices, and a greater willingness to disclose sensitive personal information to them. Our work furthermore shows that voice cloning leads to homogenization of speaker characteristics, as measured by reduced variance in accent, speaking rate, and the audio embedding space. Together, our results highlight a new set of limitations and risks of voice cloning technology and their potential impact on human behavior.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 25 of its 57 references that are in the map. First posted on arXiv 15 May 2026.
Builds on
25 of 57 references.
References · 25
- 2022Robust Speech Recognition via Large-Scale Weak Supervisioncited by 4,057
- 2023Neural Codec Language Models are Zero-Shot Text to Speech Synthesizerscited by 752
- 2020ECAPA-TDNN: Emphasized Channel Attention, Propagation and Aggregation in TDNN Based Speaker Verificationcited by 628
- 2023Voicebox: Text-Guided Multilingual Universal Speech Generation at Scalecited by 379
- 2024Seed-TTS: A Family of High-Quality Versatile Speech Generation Modelscited by 300
- 2023Self-Consuming Generative Models Go MADcited by 250
- 2023NaturalSpeech 2: Latent Diffusion Models are Natural and Zero-Shot Speech and Singing Synthesizerscited by 235
- 2024NaturalSpeech 3: Zero-Shot Speech Synthesis with Factorized Codec and Diffusion Modelscited by 226
- 2022PromptTTS: Controllable Text-to-Speech with Text Descriptionscited by 110
- 2018Neural Voice Cloning with a Few Samplescited by 100
- 2023InstructTTS: Modelling Expressive TTS in Discrete Latent Space with Natural Language Style Promptcited by 92
- 2023Warning: Humans Cannot Reliably Detect Speech Deepfakescited by 20
13 more
- 2023CommonAccent: Exploring Large Acoustic Pretrained Models for Accent Classification Based on Common Voicecited by 20
- 2024Not My Voice! A Taxonomy of Ethical and Safety Harms of Speech Generatorscited by 18
- 2024Transparent AI Disclosure Obligations: Who, What, When, Where, Why, Howcited by 17
- 2024ControlSpeech: Towards Simultaneous and Independent Zero-shot Speaker Cloning and Zero-shot Language Style Controlcited by 17
- 2024Rel-A.I.: An Interaction-Centered Approach To Measuring Human-LM Reliancecited by 16
- 2024EmoKnob: Enhance Voice Cloning with Fine-Grained Emotion Controlcited by 13
- 2024Generating Data with Text-to-Speech and Large-Language Models for Conversational Speech Recognitioncited by 9
- 2025"It's not a representation of me": Examining Accent Bias and Digital Exclusion in Synthetic AI Voice Servicescited by 9
- 2025Audio2Face-3D: Audio-driven Realistic Facial Animation For Digital Avatarscited by 7
- 2023Learning Through AI-Clones: Enhancing Self-Perception and Presentation Performancecited by 4
- 2024Advancing Speech Language Models by Scaling Supervised Fine-Tuning with Over 60,000 Hours of Synthetic Speech Dialogue Datacited by 3
- 2026AI Twin: Enhancing ESL Speaking Practice through AI Self-Clones of a Better Mecited by 2
- 2024Preset-Voice Matching for Privacy Regulated Speech-to-Speech Translation Systemscited by 1
Built on by
Not cited yet.