# Detecting Voice Cloning Attacks via Timbre Watermarking

arXiv: https://arxiv.org/abs/2312.03410 · HTML: https://mlsanity.com/paper/2312.03410/

## Key facts

- Cited by 44 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 6 Dec 2023.

## Built on by (most cited first, 30 of 44)

- [SoK: Watermarking for AI-Generated Content](https://mlsanity.com/paper/2411.18479/) (2024)
- [GS-Hider: Hiding Messages into 3D Gaussian Splatting](https://mlsanity.com/paper/2405.15118/) (2024)
- [AudioMarkBench: Benchmarking Robustness of Audio Watermarking](https://mlsanity.com/paper/2406.06979/) (2024)
- [GROOT: Generating Robust Watermark for Diffusion-Model-Based Audio Synthesis](https://mlsanity.com/paper/2407.10471/) (2024)
- [SecureGS: Boosting the Security and Fidelity of 3D Gaussian Splatting Steganography](https://mlsanity.com/paper/2503.06118/) (2025)
- [A Comprehensive Real-World Assessment of Audio Watermarking Algorithms: Will They Survive Neural Codecs?](https://mlsanity.com/paper/2505.19663/) (2025)
- [SoK: How Robust is Audio Watermarking in Generative AI models?](https://mlsanity.com/paper/2503.19176/) (2025)
- [VoiceShop: A Unified Speech-to-Speech Framework for Identity-Preserving Zero-Shot Voice Editing](https://mlsanity.com/paper/2404.06674/) (2024)
- [U Can't Gen This? A Survey of Intellectual Property Protection Methods for Data in Generative AI](https://mlsanity.com/paper/2406.15386/) (2024)
- [AI-generated Image Detection: Passive or Watermark?](https://mlsanity.com/paper/2411.13553/) (2024)
- [SafeSpeech: Robust and Universal Voice Protection Against Malicious Speech Synthesis](https://mlsanity.com/paper/2504.09839/) (2025)
- [XAttnMark: Learning Robust Audio Watermarking with Cross-Attention](https://mlsanity.com/paper/2502.04230/) (2025)
- [Securing the Future of GenAI: Policy and Technology](https://mlsanity.com/paper/2407.12999/) (2024)
- [Self Voice Conversion as an Attack against Neural Audio Watermarking](https://mlsanity.com/paper/2601.20432/) (2026)
- [VoiceMark: Zero-Shot Voice Cloning-Resistant Watermarking Approach Leveraging Speaker-Specific Latents](https://mlsanity.com/paper/2505.21568/) (2025)
- [HarmonicAttack: An Adaptive Cross-Domain Audio Watermark Removal](https://mlsanity.com/paper/2511.21577/) (2025)
- [De-AntiFake: Rethinking the Protective Perturbations Against Voice Cloning Attacks](https://mlsanity.com/paper/2507.02606/) (2025)
- [The Impact of Audio Watermarking on Audio Anti-Spoofing Countermeasures](https://mlsanity.com/paper/2509.20736/) (2025)
- [Hidden in Plain Tokens: Simply Robust, Gradient-Free Watermark for Synthetic Audio](https://mlsanity.com/paper/2605.25967/) (2026)
- [Preset-Voice Matching for Privacy Regulated Speech-to-Speech Translation Systems](https://mlsanity.com/paper/2407.13153/) (2024)
- [Exploiting Vulnerabilities in Speech Translation Systems through Targeted Adversarial Attacks](https://mlsanity.com/paper/2503.00957/) (2025)
- [SOLIDO: A Robust Watermarking Method for Speech Synthesis via Low-Rank Adaptation](https://mlsanity.com/paper/2504.15035/) (2025)
- [CloneShield: A Framework for Universal Perturbation Against Zero-Shot Voice Cloning](https://mlsanity.com/paper/2505.19119/) (2025)
- [EchoMark: Perceptual Acoustic Environment Transfer with Watermark-Embedded Room Impulse Response](https://mlsanity.com/paper/2511.06458/) (2025)
- [VocBulwark: Towards Practical Generative Speech Watermarking via Additional-Parameter Injection](https://mlsanity.com/paper/2601.22556/) (2026)
- [Latent-Mark: An Audio Watermark Robust to Neural Codec Compression](https://mlsanity.com/paper/2603.05310/) (2026)
- [MelShield: Robust Mel-Domain Audio Watermarking for Provenance Attribution of AI Generated Synthesized Speech](https://mlsanity.com/paper/2605.01515/) (2026)
- [The Watermark Shortcut: How Provenance Marking Sabotages Audio Deepfake Detection](https://mlsanity.com/paper/2606.23335/) (2026)
- [CRAW: Codec Robust Audio Watermarking](https://mlsanity.com/paper/2609.03107/) (2026)
- [Speech Watermarking with Discrete Intermediate Representations](https://mlsanity.com/paper/2412.13917/) (2024)

## Abstract (arXiv)

Nowadays, it is common to release audio content to the public. However, with the rise of voice cloning technology, attackers have the potential to easily impersonate a specific person by utilizing his publicly released audio without any permission. Therefore, it becomes significant to detect any potential misuse of the released audio content and protect its timbre from being impersonated. To this end, we introduce a novel concept, "Timbre Watermarking", which embeds watermark information into the target individual's speech, eventually defeating the voice cloning attacks. To ensure the watermark is robust to the voice cloning model's learning process, we design an end-to-end voice cloning-resistant detection framework. The core idea of our solution is to embed and extract the watermark in the frequency domain in a temporally invariant manner. To acquire generalization across different voice cloning attacks, we modulate their shared process and integrate it into our framework as a distortion layer. Experiments demonstrate that the proposed timbre watermarking can defend against different voice cloning attacks, exhibit strong resistance against various adaptive attacks (e.g., reconstruction-based removal attacks, watermark overwriting attacks), and achieve practicality in real-world services such as PaddleSpeech, Voice-Cloning-App, and so-vits-svc. In addition, ablation studies are also conducted to verify the effectiveness of our design. Some audio samples are available at https://timbrewatermarking.github.io/samples.