# Single and Multi-Speaker Cloned Voice Detection: From Perceptual to Learned Features

arXiv: https://arxiv.org/abs/2307.07683 · HTML: https://mlsanity.com/paper/2307.07683/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- Builds on 9 of its 30 references that are in the map.
- First posted on arXiv 15 Jul 2023.

## Builds on (most cited first, 9 of 9)

- [YourTTS: Towards Zero-Shot Multi-Speaker TTS and Zero-Shot Voice Conversion for everyone](https://mlsanity.com/paper/2112.02418/) (2021)
- [End-to-end anti-spoofing with RawNet2](https://mlsanity.com/paper/2011.01108/) (2020)
- [An Overview of Voice Conversion and its Challenges: From Statistical Modeling to Deep Learning](https://mlsanity.com/paper/2008.03648/) (2020)
- [ADD 2022: the First Audio Deep Synthesis Detection Challenge](https://mlsanity.com/paper/2202.08433/) (2022)
- [WaveFake: A Data Set to Facilitate Audio Deepfake Detection](https://mlsanity.com/paper/2111.02813/) (2021)
- [TitaNet: Neural Model for speaker representation with 1D Depth-wise separable convolutions and global context](https://mlsanity.com/paper/2110.04410/) (2021)
- [An Initial Investigation for Detecting Vocoder Fingerprints of Fake Audio](https://mlsanity.com/paper/2208.09646/) (2022)
- [Deepfake audio detection by speaker verification](https://mlsanity.com/paper/2209.14098/) (2022)
- [Combining Automatic Speaker Verification and Prosody Analysis for Synthetic Speech Detection](https://mlsanity.com/paper/2210.17222/) (2022)

## Built on by (most cited first, 10 of 10)

- [Proactive Detection of Voice Cloning with Localized Watermarking](https://mlsanity.com/paper/2401.17264/) (2024)
- [SONICS: Synthetic Or Not -- Identifying Counterfeit Songs](https://mlsanity.com/paper/2408.14080/) (2024)
- [People are poorly equipped to detect AI-powered voice clones](https://mlsanity.com/paper/2410.03791/) (2024)
- [Harder or Different? Understanding Generalization of Audio Deepfake Detection](https://mlsanity.com/paper/2406.03512/) (2024)
- [Training-Free Deepfake Voice Recognition by Leveraging Large-Scale Pre-Trained Models](https://mlsanity.com/paper/2405.02179/) (2024)
- [The DeepSpeak Dataset](https://mlsanity.com/paper/2408.05366/) (2024)
- [Improving Generalization for AI-Synthesized Voice Detection](https://mlsanity.com/paper/2412.19279/) (2024)
- [Human Action CLIPs: Detecting AI-generated Human Motion](https://mlsanity.com/paper/2412.00526/) (2024)
- [Personhood Credentials: Human-Centered Design Recommendation Balancing Security, Usability, and Trust](https://mlsanity.com/paper/2502.16375/) (2025)
- [Acoustic Cybersecurity: Exploiting Voice-Activated Systems](https://mlsanity.com/paper/2312.00039/) (2023)

## Abstract (arXiv)

Synthetic-voice cloning technologies have seen significant advances in recent years, giving rise to a range of potential harms. From small- and large-scale financial fraud to disinformation campaigns, the need for reliable methods to differentiate real and synthesized voices is imperative. We describe three techniques for differentiating a real from a cloned voice designed to impersonate a specific person. These three approaches differ in their feature extraction stage with low-dimensional perceptual features offering high interpretability but lower accuracy, to generic spectral features, and end-to-end learned features offering less interpretability but higher accuracy. We show the efficacy of these approaches when trained on a single speaker's voice and when trained on multiple voices. The learned features consistently yield an equal error rate between 0% and 4%, and are reasonably robust to adversarial laundering.