# V.O.I.C.E (Voice, Ownership, Identity, Control, Expression): Risk Taxonomy of Synthetic Voice Generation From Empirical Data

arXiv: https://arxiv.org/abs/2604.24794 · HTML: https://mlsanity.com/paper/2604.24794/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 32 of its 84 references that are in the map.
- First posted on arXiv 25 Apr 2026.

## Builds on (most cited first, 30 of 32)

- [Neural Codec Language Models are Zero-Shot Text to Speech Synthesizers](https://mlsanity.com/paper/2301.02111/) (2023)
- [VoxCeleb: a large-scale speaker identification dataset](https://mlsanity.com/paper/1706.08612/) (2017)
- [Libri-Light: A Benchmark for ASR with Limited or No Supervision](https://mlsanity.com/paper/1912.07875/) (2019)
- [GigaSpeech: An Evolving, Multi-domain ASR Corpus with 10,000 Hours of Transcribed Audio](https://mlsanity.com/paper/2106.06909/) (2021)
- [Voicebox: Text-Guided Multilingual Universal Speech Generation at Scale](https://mlsanity.com/paper/2306.15687/) (2023)
- [Transfer Learning from Speaker Verification to Multispeaker Text-To-Speech Synthesis](https://mlsanity.com/paper/1806.04558/) (2018)
- [YourTTS: Towards Zero-Shot Multi-Speaker TTS and Zero-Shot Voice Conversion for everyone](https://mlsanity.com/paper/2112.02418/) (2021)
- [The Creation and Detection of Deepfakes: A Survey](https://mlsanity.com/paper/2004.11138/) (2020)
- [The Malicious Use of Artificial Intelligence: Forecasting, Prevention, and Mitigation](https://mlsanity.com/paper/1802.07228/) (2018)
- [NaturalSpeech: End-to-End Text to Speech Synthesis with Human-Level Quality](https://mlsanity.com/paper/2205.04421/) (2022)
- [Sociotechnical Harms of Algorithmic Systems: Scoping a Taxonomy for Harm Reduction](https://mlsanity.com/paper/2210.05791/) (2022)
- [Does Audio Deepfake Detection Generalize?](https://mlsanity.com/paper/2203.16263/) (2022)
- [Qwen3-TTS Technical Report](https://mlsanity.com/paper/2601.15621/) (2026)
- [Deep Voice 2: Multi-Speaker Neural Text-to-Speech](https://mlsanity.com/paper/1705.08947/) (2017)
- [Fish-Speech: Leveraging Large Language Models for Advanced Multilingual Text-to-Speech Synthesis](https://mlsanity.com/paper/2411.01156/) (2024)
- [Autoregressive Diffusion Transformer for Text-to-Speech Synthesis](https://mlsanity.com/paper/2406.05551/) (2024)
- [OpenVoice: Versatile Instant Voice Cloning](https://mlsanity.com/paper/2312.01479/) (2023)
- [AVQVC: One-shot Voice Conversion by Vector Quantization with applying contrastive learning](https://mlsanity.com/paper/2202.10020/) (2022)
- [Warning: Humans Cannot Reliably Detect Speech Deepfakes](https://mlsanity.com/paper/2301.07829/) (2023)
- [LM-VC: Zero-shot Voice Conversion via Speech Generation based on Language Models](https://mlsanity.com/paper/2306.10521/) (2023)
- [Not My Voice! A Taxonomy of Ethical and Safety Harms of Speech Generators](https://mlsanity.com/paper/2402.01708/) (2024)
- [Aligning AI with Public Values: Deliberation and Decision-Making for Governing Multimodal LLMs in Political Video Analysis](https://mlsanity.com/paper/2410.01817/) (2024)
- [An Overview & Analysis of Sequence-to-Sequence Emotional Voice Conversion](https://mlsanity.com/paper/2203.15873/) (2022)
- [A Hierarchical Speaker Representation Framework for One-shot Singing Voice Conversion](https://mlsanity.com/paper/2206.13762/) (2022)
- [SafeSpeech: Robust and Universal Voice Protection Against Malicious Speech Synthesis](https://mlsanity.com/paper/2504.09839/) (2025)
- [Exploring Parent-Child Perceptions on Safety in Generative AI: Concerns, Mitigation Strategies, and Design Implications](https://mlsanity.com/paper/2406.10461/) (2024)
- [TacoLM: GaTed Attention Equipped Codec Language Model are Efficient Zero-Shot Text to Speech Synthesizers](https://mlsanity.com/paper/2406.15752/) (2024)
- [Improving Audio Codec-based Zero-Shot Text-to-Speech Synthesis with Multi-Modal Context and Large Language Model](https://mlsanity.com/paper/2406.03706/) (2024)
- [Sound Check: Auditing Audio Datasets](https://mlsanity.com/paper/2410.13114/) (2024)
- [Human perception of audio deepfakes: the role of language and speaking style](https://mlsanity.com/paper/2512.09221/) (2025)

## Abstract (arXiv)

As generative voice models are rapidly advancing in both capabilities and public utilization, the unconsented collection, reuse, and synthesis of voice data are introducing new classes of privacy, security and governance risk that are poorly captured by existing, largely uniform threat models. To fill the gap, we present V.O.I.C.E, a taxonomy of voice generation risk grounded in a multi-source threat modeling effort with 569 incidents from major AI incident database, FTC and Internet Crime Complaint Center (IC3); 1067 direct incident reports from U.S. based participants across diverse groups (including voice actors, internet personalities, political personnel, and general public); and 2,221 Reddit discussions. Grounded in real-world data, our taxonomy explicitly models how risk emerges, interact with contextual factors such as degree of exposure, social visibility, and the availability of legal protections for various affected groups.