# Audio Codec Augmentation for Robust Collaborative Watermarking of Speech Synthesis

arXiv: https://arxiv.org/abs/2409.13382 · HTML: https://mlsanity.com/paper/2409.13382/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 24 of its 39 references that are in the map.
- First posted on arXiv 20 Sep 2024.

## Builds on (most cited first, 24 of 24)

- [Neural Discrete Representation Learning](https://mlsanity.com/paper/1711.00937/) (2017)
- [Estimating or Propagating Gradients Through Stochastic Neurons for Conditional Computation](https://mlsanity.com/paper/1308.3432/) (2013)
- [HiFi-GAN: Generative Adversarial Networks for Efficient and High Fidelity Speech Synthesis](https://mlsanity.com/paper/2010.05646/) (2020)
- [SoundStream: An End-to-End Neural Audio Codec](https://mlsanity.com/paper/2107.03312/) (2021)
- [Neural Codec Language Models are Zero-Shot Text to Speech Synthesizers](https://mlsanity.com/paper/2301.02111/) (2023)
- [High Fidelity Neural Audio Compression](https://mlsanity.com/paper/2210.13438/) (2022)
- [High-Fidelity Audio Compression with Improved RVQGAN](https://mlsanity.com/paper/2306.06546/) (2023)
- [Transfer Learning from Speaker Verification to Multispeaker Text-To-Speech Synthesis](https://mlsanity.com/paper/1806.04558/) (2018)
- [YourTTS: Towards Zero-Shot Multi-Speaker TTS and Zero-Shot Voice Conversion for everyone](https://mlsanity.com/paper/2112.02418/) (2021)
- [Does Audio Deepfake Detection Generalize?](https://mlsanity.com/paper/2203.16263/) (2022)
- [ASVspoof 2021: Towards Spoofed and Deepfake Speech Detection in the Wild](https://mlsanity.com/paper/2210.02437/) (2022)
- [Automatic speaker verification spoofing and deepfake detection using wav2vec 2.0 and data augmentation](https://mlsanity.com/paper/2202.12233/) (2022)
- [LibriTTS-R: A Restored Multi-Speaker Text-to-Speech Corpus](https://mlsanity.com/paper/2305.18802/) (2023)
- [ADD 2022: the First Audio Deep Synthesis Detection Challenge](https://mlsanity.com/paper/2202.08433/) (2022)
- [Responsible Disclosure of Generative Models Using Scalable Fingerprinting](https://mlsanity.com/paper/2012.08726/) (2020)
- [Proactive Detection of Voice Cloning with Localized Watermarking](https://mlsanity.com/paper/2401.17264/) (2024)
- [WavMark: Watermarking for Audio Generation](https://mlsanity.com/paper/2308.12770/) (2023)
- [Speech is Silver, Silence is Golden: What do ASVspoof-trained Models Really Learn?](https://mlsanity.com/paper/2106.12914/) (2021)
- [TorchAudio 2.1: Advancing speech recognition, self-supervised learning, and audio processing components for PyTorch](https://mlsanity.com/paper/2310.17864/) (2023)
- [Removing Batch Normalization Boosts Adversarial Training](https://mlsanity.com/paper/2207.01156/) (2022)
- [SilentCipher: Deep Audio Watermarking](https://mlsanity.com/paper/2406.03822/) (2024)
- [High-Quality, Low-Delay Music Coding in the Opus Codec](https://mlsanity.com/paper/1602.04845/) (2016)
- [Generalization of Spoofing Countermeasures: a Case Study with ASVspoof 2015 and BTAS 2016 Corpora](https://mlsanity.com/paper/1901.08025/) (2019)
- [Collaborative Watermarking for Adversarial Speech Synthesis](https://mlsanity.com/paper/2309.15224/) (2023)

## Built on by (most cited first, 4 of 4)

- [A Comprehensive Real-World Assessment of Audio Watermarking Algorithms: Will They Survive Neural Codecs?](https://mlsanity.com/paper/2505.19663/) (2025)
- [Self Voice Conversion as an Attack against Neural Audio Watermarking](https://mlsanity.com/paper/2601.20432/) (2026)
- [Latent-Mark: An Audio Watermark Robust to Neural Codec Compression](https://mlsanity.com/paper/2603.05310/) (2026)
- [Efficient Speech Watermarking for Speech Synthesis via Progressive Knowledge Distillation](https://mlsanity.com/paper/2509.19812/) (2025)

## Abstract (arXiv)

Automatic detection of synthetic speech is becoming increasingly important as current synthesis methods are both near indistinguishable from human speech and widely accessible to the public. Audio watermarking and other active disclosure methods of are attracting research activity, as they can complement traditional deepfake defenses based on passive detection. In both active and passive detection, robustness is of major interest. Traditional audio watermarks are particularly susceptible to removal attacks by audio codec application. Most generated speech and audio content released into the wild passes through an audio codec purely as a distribution method. We recently proposed collaborative watermarking as method for making generated speech more easily detectable over a noisy but differentiable transmission channel. This paper extends the channel augmentation to work with non-differentiable traditional audio codecs and neural audio codecs and evaluates transferability and effect of codec bitrate over various configurations. The results show that collaborative watermarking can be reliably augmented by black-box audio codecs using a waveform-domain straight-through-estimator for gradient approximation. Furthermore, that results show that channel augmentation with a neural audio codec transfers well to traditional codecs. Listening tests demonstrate collaborative watermarking incurs negligible perceptual degradation with high bitrate codecs or DAC at 8kbps.