# Self Voice Conversion as an Attack against Neural Audio Watermarking

arXiv: https://arxiv.org/abs/2601.20432 · HTML: https://mlsanity.com/paper/2601.20432/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 35 of its 45 references that are in the map.
- First posted on arXiv 28 Jan 2026.

## Builds on (most cited first, 30 of 35)

- [HuBERT: Self-Supervised Speech Representation Learning by Masked Prediction of Hidden Units](https://mlsanity.com/paper/2106.07447/) (2021)
- [WavLM: Large-Scale Self-Supervised Pre-Training for Full Stack Speech Processing](https://mlsanity.com/paper/2110.13900/) (2021)
- [HiFi-GAN: Generative Adversarial Networks for Efficient and High Fidelity Speech Synthesis](https://mlsanity.com/paper/2010.05646/) (2020)
- [Neural Codec Language Models are Zero-Shot Text to Speech Synthesizers](https://mlsanity.com/paper/2301.02111/) (2023)
- [High Fidelity Neural Audio Compression](https://mlsanity.com/paper/2210.13438/) (2022)
- [LibriTTS: A Corpus Derived from LibriSpeech for Text-to-Speech](https://mlsanity.com/paper/1904.02882/) (2019)
- [ECAPA-TDNN: Emphasized Channel Attention, Propagation and Aggregation in TDNN Based Speaker Verification](https://mlsanity.com/paper/2005.07143/) (2020)
- [Conditional Variational Autoencoder with Adversarial Learning for End-to-End Text-to-Speech](https://mlsanity.com/paper/2106.06103/) (2021)
- [High-Fidelity Audio Compression with Improved RVQGAN](https://mlsanity.com/paper/2306.06546/) (2023)
- [UTMOS: UTokyo-SaruLab System for VoiceMOS Challenge 2022](https://mlsanity.com/paper/2204.02152/) (2022)
- [WaveGlow: A Flow-based Generative Network for Speech Synthesis](https://mlsanity.com/paper/1811.00002/) (2018)
- [NaturalSpeech 2: Latent Diffusion Models are Natural and Zero-Shot Speech and Singing Synthesizers](https://mlsanity.com/paper/2304.09116/) (2023)
- [An Overview of Voice Conversion and its Challenges: From Statistical Modeling to Deep Learning](https://mlsanity.com/paper/2008.03648/) (2020)
- [CREPE: A Convolutional Representation for Pitch Estimation](https://mlsanity.com/paper/1802.06182/) (2018)
- [Proactive Detection of Voice Cloning with Localized Watermarking](https://mlsanity.com/paper/2401.17264/) (2024)
- [Voice Conversion With Just Nearest Neighbors](https://mlsanity.com/paper/2305.18975/) (2023)
- [A Comparative Study on Recent Neural Spoofing Countermeasures for Synthetic Speech Detection](https://mlsanity.com/paper/2103.11326/) (2021)
- [WavMark: Watermarking for Audio Generation](https://mlsanity.com/paper/2308.12770/) (2023)
- [Detecting Voice Cloning Attacks via Timbre Watermarking](https://mlsanity.com/paper/2312.03410/) (2023)
- [AudioMarkBench: Benchmarking Robustness of Audio Watermarking](https://mlsanity.com/paper/2406.06979/) (2024)
- [DeAR: A Deep-learning-based Audio Re-recording Resilient Watermarking](https://mlsanity.com/paper/2212.02339/) (2022)
- [SilentCipher: Deep Audio Watermarking](https://mlsanity.com/paper/2406.03822/) (2024)
- [GROOT: Generating Robust Watermark for Diffusion-Model-Based Audio Synthesis](https://mlsanity.com/paper/2407.10471/) (2024)
- [The VoicePrivacy 2022 Challenge Evaluation Plan](https://mlsanity.com/paper/2203.12468/) (2022)
- [A Comprehensive Real-World Assessment of Audio Watermarking Algorithms: Will They Survive Neural Codecs?](https://mlsanity.com/paper/2505.19663/) (2025)
- [Neural Harmonic-plus-Noise Waveform Model with Trainable Maximum Voice Frequency for Text-to-Speech Synthesis](https://mlsanity.com/paper/1908.10256/) (2019)
- [Deep Audio Watermarks are Shallow: Limitations of Post-Hoc Watermarking Techniques for Speech](https://mlsanity.com/paper/2504.10782/) (2025)
- [SoK: How Robust is Audio Watermarking in Generative AI models?](https://mlsanity.com/paper/2503.19176/) (2025)
- [Who is Speaking Actually? Robust and Versatile Speaker Traceability for Voice Conversion](https://mlsanity.com/paper/2305.05152/) (2023)
- [XAttnMark: Learning Robust Audio Watermarking with Cross-Attention](https://mlsanity.com/paper/2502.04230/) (2025)

## Built on by (most cited first, 4 of 4)

- [Latent-Mark: An Audio Watermark Robust to Neural Codec Compression](https://mlsanity.com/paper/2603.05310/) (2026)
- [The Watermark Shortcut: How Provenance Marking Sabotages Audio Deepfake Detection](https://mlsanity.com/paper/2606.23335/) (2026)
- [What Counts as Real? Speech Restoration and Voice Quality Conversion Pose New Challenges to Deepfake Detection](https://mlsanity.com/paper/2603.14033/) (2026)
- [From Scores to Evidence: Auditable Decisions Can Improve Speech Deepfake Detection](https://mlsanity.com/paper/2609.08899/) (2026)

## Abstract (arXiv)

Audio watermarking embeds auxiliary information into speech while maintaining speaker identity, linguistic content, and perceptual quality. Although recent advances in neural and digital signal processing-based watermarking methods have improved imperceptibility and embedding capacity, robustness is still primarily assessed against conventional distortions such as compression, additive noise, and resampling. However, the rise of deep learning-based attacks introduces novel and significant threats to watermark security. In this work, we investigate self voice conversion as a universal, content-preserving attack against audio watermarking systems. Self voice conversion remaps a speaker's voice to the same identity while altering acoustic characteristics through a voice conversion model. We demonstrate that this attack severely degrades the reliability of state-of-the-art watermarking approaches and highlight its implications for the security of modern audio watermarking techniques.