# The Watermark Shortcut: How Provenance Marking Sabotages Audio Deepfake Detection

arXiv: https://arxiv.org/abs/2606.23335 · HTML: https://mlsanity.com/paper/2606.23335/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Placed in Watermarking generated content and models, one of 12 directions in LLM safety and security.
- Builds on 30 of its 37 references that are in the map.
- First posted on arXiv 22 Jun 2026.

## Placed in

- LLM safety and security / [Watermarking generated content and models](https://mlsanity.com/safety/watermarking-generated-content-and-models/)

## Builds on (most cited first, 30 of 30)

- [Shortcut Learning in Deep Neural Networks](https://mlsanity.com/paper/2004.07780/) (2020)
- [Natural TTS Synthesis by Conditioning WaveNet on Mel Spectrogram Predictions](https://mlsanity.com/paper/1712.05884/) (2017)
- [Neural Codec Language Models are Zero-Shot Text to Speech Synthesizers](https://mlsanity.com/paper/2301.02111/) (2023)
- [Conditional Variational Autoencoder with Adversarial Learning for End-to-End Text-to-Speech](https://mlsanity.com/paper/2106.06103/) (2021)
- [F5-TTS: A Fairytaler that Fakes Fluent and Faithful Speech with Flow Matching](https://mlsanity.com/paper/2410.06885/) (2024)
- [CosyVoice: A Scalable Multilingual Zero-shot Text-to-speech Synthesizer based on Supervised Semantic Tokens](https://mlsanity.com/paper/2407.05407/) (2024)
- [AASIST: Audio Anti-Spoofing using Integrated Spectro-Temporal Graph Attention Networks](https://mlsanity.com/paper/2110.01200/) (2021)
- [ASVspoof 2021: accelerating progress in spoofed and deepfake speech detection](https://mlsanity.com/paper/2109.00537/) (2021)
- [Does Audio Deepfake Detection Generalize?](https://mlsanity.com/paper/2203.16263/) (2022)
- [Proactive Detection of Voice Cloning with Localized Watermarking](https://mlsanity.com/paper/2401.17264/) (2024)
- [WavMark: Watermarking for Audio Generation](https://mlsanity.com/paper/2308.12770/) (2023)
- [Detecting Voice Cloning Attacks via Timbre Watermarking](https://mlsanity.com/paper/2312.03410/) (2023)
- [Speech is Silver, Silence is Golden: What do ASVspoof-trained Models Really Learn?](https://mlsanity.com/paper/2106.12914/) (2021)
- [AudioMarkBench: Benchmarking Robustness of Audio Watermarking](https://mlsanity.com/paper/2406.06979/) (2024)
- [Circumventing shortcuts in audio-visual deepfake detection datasets with unsupervised learning](https://mlsanity.com/paper/2412.00175/) (2024)
- [SilentCipher: Deep Audio Watermarking](https://mlsanity.com/paper/2406.03822/) (2024)
- [Are Watermarks Bugs for Deepfake Detectors? Rethinking Proactive Forensics](https://mlsanity.com/paper/2404.17867/) (2024)
- [A temporal chrominance trigger for clean-label backdoor attack against anti-spoof rebroadcast detection](https://mlsanity.com/paper/2206.01102/) (2022)
- [SoK: How Robust is Audio Watermarking in Generative AI models?](https://mlsanity.com/paper/2503.19176/) (2025)
- [Harder or Different? Understanding Generalization of Audio Deepfake Detection](https://mlsanity.com/paper/2406.03512/) (2024)
- [Dataset artefacts in anti-spoofing systems: a case study on the ASVspoof 2017 benchmark](https://mlsanity.com/paper/2010.07913/) (2020)
- [Yours or Mine? Overwriting Attacks Against Neural Audio Watermarking](https://mlsanity.com/paper/2509.05835/) (2025)
- [Self Voice Conversion as an Attack against Neural Audio Watermarking](https://mlsanity.com/paper/2601.20432/) (2026)
- [VoiceMark: Zero-Shot Voice Cloning-Resistant Watermarking Approach Leveraging Speaker-Specific Latents](https://mlsanity.com/paper/2505.21568/) (2025)
- [WaveVerify: A Novel Audio Watermarking Framework for Media Authentication and Combatting Deepfakes](https://mlsanity.com/paper/2507.21150/) (2025)
- [HarmonicAttack: An Adaptive Cross-Domain Audio Watermark Removal](https://mlsanity.com/paper/2511.21577/) (2025)
- [RobustSora: De-Watermarked Benchmark for Robust AI-Generated Video Detection](https://mlsanity.com/paper/2512.10248/) (2025)
- [The Impact of Audio Watermarking on Audio Anti-Spoofing Countermeasures](https://mlsanity.com/paper/2509.20736/) (2025)
- [FakeMark: Deepfake Speech Attribution With Watermarked Artifacts](https://mlsanity.com/paper/2510.12042/) (2025)
- [Bloodroot: When Watermarking Turns Poisonous For Stealthy Backdoor](https://mlsanity.com/paper/2510.07909/) (2025)

## Built on by (most cited first, 1 of 1)

- [From Scores to Evidence: Auditable Decisions Can Improve Speech Deepfake Detection](https://mlsanity.com/paper/2609.08899/) (2026)

## Abstract (arXiv)

Provenance watermarking is increasingly treated as a safeguard for synthetic speech, whether built directly into speech-generation models such as Chatterbox, provided through dedicated techniques such as AudioSeal, or deployed by commercial platforms such as ElevenLabs. We identify a previously uncharacterized liability: when synthetic speech is watermarked and human speech is not, detectors trained alongside latch onto the watermark as a spurious "watermark => fake" shortcut. This single feature yields three coupled failures: generalization degradation (model performance deteriorates on unseen data), strip-to-evade (a watermarked fake escapes once unwatermarked), and mark-to-frame (watermarking a real voice flags it as fake). In a controlled white-box experiment, a watermark-trained detector shows all three (for example, mark-to-frame lifts Equal Error Rate from 16% to 75%). In a black-box test of a commercial API, we show that adding a watermark to real speech disguises it as fake. However, this shortcut is fixable: retraining with the watermark on both classes decorrelates it and restores clean behavior. We release experiment data as a paired clean-versus-watermarked corpus (WASP).