mlsanityData to 7 Oct 2026

Proactive Detection of Voice Cloning with Localized Watermarking

arXiv 2401.17264 ↗First posted 30 Jan 2024cs.SD cs.AI cs.CRcited by 67

Not on the map. Why?

Abstract (arXiv)

In the rapidly evolving field of speech generative models, there is a pressing need to ensure audio authenticity against the risks of voice cloning. We present AudioSeal, the first audio watermarking technique designed specifically for localized detection of AI-generated speech. AudioSeal employs a generator/detector architecture trained jointly with a localization loss to enable localized watermark detection up to the sample level, and a novel perceptual loss inspired by auditory masking, that enables AudioSeal to achieve better imperceptibility. AudioSeal achieves state-of-the-art performance in terms of robustness to real life audio manipulations and imperceptibility based on automatic and human evaluation metrics. Additionally, AudioSeal is designed with a fast, single-pass detector, that significantly surpasses existing models in speed - achieving detection up to two orders of magnitude faster, making it ideal for large-scale and real-time applications.

Key facts. Cited by 67 papers in the map (data through 7 Oct 2026). Builds on 43 of its 74 references that are in the map. First posted on arXiv 30 Jan 2024.

Builds on

43 of 74 references.

References · 43

  1. 2016WaveNet: A Generative Model for Raw Audiocited by 1,710
  2. 2020HiFi-GAN: Generative Adversarial Networks for Efficient and High Fidelity Speech Synthesiscited by 1,139
  3. 2021SoundStream: An End-to-End Neural Audio Codeccited by 756
  4. 2023Neural Codec Language Models are Zero-Shot Text to Speech Synthesizerscited by 737
  5. 2022High Fidelity Neural Audio Compressioncited by 673
  6. 2023A Watermark for Large Language Modelscited by 667
  7. 2022AudioLM: a Language Modeling Approach to Audio Generationcited by 611
  8. 2021Conditional Variational Autoencoder with Adversarial Learning for End-to-End Text-to-Speechcited by 555
  9. 2023Simple and Controllable Music Generationcited by 480
  10. 2021VoxPopuli: A Large-Scale Multilingual Speech Corpus for Representation Learning, Semi-Supervised Learning and Interpretationcited by 455
  11. 2023High-Fidelity Audio Compression with Improved RVQGANcited by 423
  12. 2018Conv-TasNet: Surpassing Ideal Time-Frequency Magnitude Masking for Speech Separationcited by 400
31 more
  1. 2023Voicebox: Text-Guided Multilingual Universal Speech Generation at Scalecited by 373
  2. 2019MelGAN: Generative Adversarial Networks for Conditional Waveform Synthesiscited by 351
  3. 2022AudioGen: Textually Guided Audio Generationcited by 292
  4. 2023The Stable Signature: Rooting Watermarks in Latent Diffusion Modelscited by 289
  5. 2021YourTTS: Towards Zero-Shot Multi-Speaker TTS and Zero-Shot Voice Conversion for everyonecited by 277
  6. 2023NaturalSpeech 2: Latent Diffusion Models are Natural and Zero-Shot Speech and Singing Synthesizerscited by 233
  7. 2019Understanding Straight-Through Estimator in Training Activation Quantized Neural Netscited by 225
  8. 2020Artificial Fingerprinting for Generative Models: Rooting Deepfake Attribution in Training Datacited by 200
  9. 2020Real Time Speech Enhancement in the Waveform Domaincited by 183
  10. 2019Dual-path RNN: efficient long sequence modeling for time-domain single-channel speech separationcited by 182
  11. 2021FakeAVCeleb: A Novel Audio-Video Multimodal Deepfake Datasetcited by 179
  12. 2023Tree-Ring Watermarks: Fingerprints for Diffusion Images that are Invisible and Robustcited by 179
  13. 2023Speak, Read and Prompt: High-Fidelity Text-to-Speech with Minimal Supervisioncited by 167
  14. 2022Does Audio Deepfake Detection Generalize?cited by 129
  15. 2023Audiobox: Unified Audio Generation with Natural Language Promptscited by 120
  16. 2022ASVspoof 2021: Towards Spoofed and Deepfake Speech Detection in the Wildcited by 118
  17. 2023SoundStorm: Efficient Parallel Audio Generationcited by 111
  18. 2018Neural Voice Cloning with a Few Samplescited by 100
  19. 2023EXPRESSO: A Benchmark and Analysis of Discrete Expressive Speech Resynthesiscited by 93
  20. 2020Responsible Disclosure of Generative Models Using Scalable Fingerprintingcited by 90
  21. 2021TorchAudio: Building Blocks for Audio and Speech Processingcited by 83
  22. 2023Three Bricks to Consolidate Watermarks for Large Language Modelscited by 80
  23. 2023WOUAF: Weight Modulation for User Attribution and Fingerprinting in Text-to-Image Diffusion Modelscited by 51
  24. 2020A Spectral Energy Distance for Parallel Speech Synthesiscited by 42
  25. 2023WavMark: Watermarking for Audio Generationcited by 41
  26. 2022A3T: Alignment-Aware Acoustic and Text Pretraining for Speech Synthesis and Editingcited by 40
  27. 2022DeAR: A Deep-learning-based Audio Re-recording Resilient Watermarkingcited by 15
  28. 2023Single and Multi-Speaker Cloned Voice Detection: From Perceptual to Learned Featurescited by 10
  29. 2023Collaborative Watermarking for Adversarial Speech Synthesiscited by 6
  30. 2023Who is Speaking Actually? Robust and Versatile Speaker Traceability for Voice Conversioncited by 5
  31. 2019Audio Watermarking over the Air With Modulated Self-Correlationcited by 1

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 18; 2025: 28; 2026: 21.

67 papers.

Sort:

By area:

  1. 2024VoiceCraft: Zero-Shot Speech Editing and Text-to-Speech in the Wildcited by 118
  2. 2024Watermark Anything with Localized Messagescited by 49
  3. 2024Video Seal: Open and Efficient Video Watermarkingcited by 45
  4. 2024SoK: Watermarking for AI-Generated Contentcited by 43
  5. 2024OmniGuard: Hybrid Manipulation Localization via Augmented Versatile Deep Image Watermarkingcited by 35
  6. 2024V2A-Mark: Versatile Deep Visual-Audio Watermarking for Manipulation Localization and Copyright Protectioncited by 27
  7. 2024GS-Hider: Hiding Messages into 3D Gaussian Splattingcited by 26
  8. 2025SynthID-Image: Image watermarking at internet scalecited by 26
  9. 2025Streaming Sequence-to-Sequence Learning with Delayed Streams Modelingcited by 24
  10. 2024AudioMarkBench: Benchmarking Robustness of Audio Watermarkingcited by 19
  11. 2024Images that Sound: Composing Images and Sounds on a Single Canvascited by 17
  12. 2025Watermarking Autoregressive Image Generationcited by 15
38 more
  1. 2025SecureGS: Boosting the Security and Fidelity of 3D Gaussian Splatting Steganographycited by 14
  2. 2025Survey on AI-Generated Media Detection: From Non-MLLM to MLLMcited by 12
  3. 2024Optimizing Adaptive Attacks against Watermarks for Language Modelscited by 11
  4. 2024Steganalysis on Digital Watermarking: Is Your Defense Truly Impervious?cited by 10
  5. 2025A Comprehensive Real-World Assessment of Audio Watermarking Algorithms: Will They Survive Neural Codecs?cited by 10
  6. 2024Meta Learning Text-to-Speech Synthesis in over 7000 Languagescited by 9
  7. 2025SoloSpeech: Enhancing Intelligibility and Quality in Target Speech Extraction through a Cascaded Generative Pipelinecited by 9
  8. 2024VoiceShop: A Unified Speech-to-Speech Framework for Identity-Preserving Zero-Shot Voice Editingcited by 7
  9. 2024AI-generated Image Detection: Passive or Watermark?cited by 7
  10. 2025SoK: How Robust is Audio Watermarking in Generative AI models?cited by 7
  11. 2024Textless Acoustic Model with Self-Supervised Distillation for Noise-Robust Expressive Speech-to-Speech Translationcited by 5
  12. 2025Guidance Watermarking for Diffusion Modelscited by 5
  13. 2026Self Voice Conversion as an Attack against Neural Audio Watermarkingcited by 4
  14. 2024Audio Codec Augmentation for Robust Collaborative Watermarking of Speech Synthesiscited by 3
  15. 2025XAttnMark: Learning Robust Audio Watermarking with Cross-Attentioncited by 3
  16. 2025VoiceMark: Zero-Shot Voice Cloning-Resistant Watermarking Approach Leveraging Speaker-Specific Latentscited by 3
  17. 2026Disclosure By Design: Identity Transparency as a Behavioural Property of Conversational AI Modelscited by 3
  18. 2025De-AntiFake: Rethinking the Protective Perturbations Against Voice Cloning Attackscited by 2
  19. 2025The Impact of Audio Watermarking on Audio Anti-Spoofing Countermeasurescited by 2
  20. 2025HarmonicAttack: An Adaptive Cross-Domain Audio Watermark Removalcited by 2
  21. 2026Who Gets Flagged? The Pluralistic Evaluation Gap in AI Content Watermarkingcited by 2
  22. 2026Hidden in Plain Tokens: Simply Robust, Gradient-Free Watermark for Synthetic Audiocited by 2
  23. 2024Prevailing Research Areas for Music AI in the Era of Foundation Modelscited by 1
  24. 2024Watermarking Training Data of Music Generation Modelscited by 1
  25. 2025CloneShield: A Framework for Universal Perturbation Against Zero-Shot Voice Cloningcited by 1
  26. 2025Tell-Tale Watermarks for Explanatory Reasoning in Synthetic Media Forensicscited by 1
  27. 2025Multimodal Evaluation of Russian-language Architecturescited by 1
  28. 2026SKeDA: A Generative Watermarking Framework for Text-to-video Diffusion Modelscited by 1
  29. 2026Latent-Mark: An Audio Watermark Robust to Neural Codec Compressioncited by 1
  30. 2026The Watermark Shortcut: How Provenance Marking Sabotages Audio Deepfake Detectioncited by 1
  31. 2026CRAW: Codec Robust Audio Watermarkingcited by 1
  32. 2025Local Differences, Global Lessons: Insights from Organisation Policies for International Legislationcited by 0
  33. 2025Targeted Data Poisoning for Black-Box Audio Datasets Ownership Verificationcited by 0
  34. 2025Protecting Your Voice: Temporal-aware Robust Watermarkingcited by 0
  35. 2025SOLIDO: A Robust Watermarking Method for Speech Synthesis via Low-Rank Adaptationcited by 0
  36. 2025Unmasking Synthetic Realities in Generative AI: A Comprehensive Review of Adversarially Robust Deepfake Detection Systemscited by 0
  37. 2025Multi-bit Audio Watermarkingcited by 0
  38. 2025AWARE: Audio Watermarking with Adversarial Resistance to Editscited by 0

50 most cited of 67 All 67 (JSON)