mlsanityData to 7 Oct 2026

SoK: Watermarking for AI-Generated Content

arXiv 2411.18479 ↗First posted 27 Nov 2024cs.CR cs.AI cs.LGcited by 43

On the map: LLM safety and security → Watermarking generated content and models

Abstract (arXiv)

As the outputs of generative AI (GenAI) techniques improve in quality, it becomes increasingly challenging to distinguish them from human-created content. Watermarking schemes are a promising approach to address the problem of distinguishing between AI and human-generated content. These schemes embed hidden signals within AI-generated content to enable reliable detection. While watermarking is not a silver bullet for addressing all risks associated with GenAI, it can play a crucial role in enhancing AI safety and trustworthiness by combating misinformation and deception. This paper presents a comprehensive overview of watermarking techniques for GenAI, beginning with the need for watermarking from historical and regulatory perspectives. We formalize the definitions and desired properties of watermarking schemes and examine the key objectives and threat models for existing approaches. Practical evaluation strategies are also explored, providing insights into the development of robust watermarking techniques capable of resisting various attacks. Additionally, we review recent representative works, highlight open challenges, and discuss potential directions for this emerging field. By offering a thorough understanding of watermarking in GenAI, this work aims to guide researchers in advancing watermarking methods and applications, and support policymakers in addressing the broader implications of GenAI.

Key facts. Cited by 43 papers in the map (data through 7 Oct 2026). Placed in Watermarking generated content and models, one of 12 directions in LLM safety and security. Builds on 92 of its 147 references that are in the map. First posted on arXiv 27 Nov 2024.

Builds on

92 of 147 references.

Closest ancestors · 45

Same direction, newest first

  1. 2024An Undetectable Watermark for Generative Image Models9 Oct
  2. 2024New constructions of pseudorandom codes11 Sep
  3. 2024GROOT: Generating Robust Watermark for Diffusion-Model-Based Audio Synthesis15 Jul
  4. 2024Bileve: Securing Text Provenance in Large Language Models Against Spoofing with Bi-level Signature4 Jun
  5. 2024Watermarking Language Models for Many Adaptive Users17 May
  6. 2024MarkLLM: An Open-Source Toolkit for LLM Watermarking16 May
  7. 2024Gaussian Shading: Provable Performance-Lossless Image Watermarking for Diffusion Models7 Apr
  8. 2024A Statistical Framework of Watermarks for Large Language Models: Pivot, Detection Efficiency and Optimal Rules1 Apr
  9. 2024WaterMax: breaking the LLM watermark detectability-robustness-quality trade-off6 Mar
  10. 2024Watermark Stealing in Large Language Models29 Feb
  11. 2024Token-Specific Watermarking with Enhanced Detectability and Semantic Coherence for Large Language Models28 Feb
  12. 2024No Free Lunch in LLM Watermarking: Trade-offs in Watermarking Design Choices25 Feb
33 more
  1. 2024GumbelSoft: Diversified Language Model Watermarking via the GumbelMax-trick20 Feb
  2. 2024k-SemStamp: A Clustering-Based Semantic Watermark for Detection of Machine-Generated Text17 Feb
  3. 2024Proving membership in LLM pretraining data via data watermarks16 Feb
  4. 2024Pseudorandom Error-Correcting Codes14 Feb
  5. 2024Adaptive Text Watermark for Large Language Models25 Jan
  6. 2023Towards Optimal Statistical Watermarking13 Dec
  7. 2023A Survey of Text Watermarking in the Era of Large Language Models13 Dec
  8. 2023On the Learnability of Watermarks for Language Models7 Dec
  9. 2023Mark My Words: Analyzing and Evaluating Language Model Watermarks1 Dec
  10. 2023Watermarks in the Sand: Impossibility of Strong Watermarking for Generative Models7 Nov
  11. 2023Publicly-Detectable Watermarking for Language Models27 Oct
  12. 2023Embarrassingly Simple Text Watermarks13 Oct
  13. 2023A Resilient and Accessible Distribution-Preserving Watermark for Large Language Models11 Oct
  14. 2023Robustness of AI-Image Detectors: Fundamental Limits and Practical Attacks29 Sep
  15. 2023Unbiased Watermark for Large Language Models22 Sep
  16. 2023An Unforgeable Publicly Verifiable Watermark for Large Language Models30 Jul
  17. 2023Robust Distortion-free Watermarks for Language Models28 Jul
  18. 2023Provable Robust Watermarking for AI-Generated Text30 Jun
  19. 2023On the Reliability of Watermarks for Large Language Models7 Jun
  20. 2023Invisible Image Watermarks Are Provably Removable Using Generative AI2 Jun
  21. 2023Tree-Ring Watermarks: Fingerprints for Diffusion Images that are Invisible and Robust31 May
  22. 2023Undetectable Watermarks for Language Models25 May
  23. 2023Who Wrote this Code? Watermarking for Code Generation24 May
  24. 2023Evading Watermark based Detection of AI-Generated Content5 May
  25. 2023The Stable Signature: Rooting Watermarks in Latent Diffusion Models27 Mar
  26. 2023Paraphrasing evades detectors of AI-generated text, but retrieval is an effective defense23 Mar
  27. 2023A Recipe for Watermarking Diffusion Models17 Mar
  28. 2023Protecting Language Generation Models via Invisible Watermarking6 Feb
  29. 2023A Watermark for Large Language Models24 Jan
  30. 2022Distillation-Resistant Watermarking for Model Protection in NLP7 Oct
  31. 2022CATER: Intellectual Property Protection on Text Generation APIs via Conditional Watermarks19 Sep
  32. 2022The Ethical Need for Watermarks in Machine-Generated Language7 Sep
  33. 2022Attributable-Watermarking of Speech Generative Models17 Feb

Other references · 47

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  3. 2020Denoising Diffusion Probabilistic Modelscited by 20,851
  4. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
  5. 2018The Unreasonable Effectiveness of Deep Features as a Perceptual Metriccited by 11,029
  6. 2020Denoising Diffusion Implicit Modelscited by 9,655
  7. 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,393
  8. 2017Towards Deep Learning Models Resistant to Adversarial Attackscited by 6,326
  9. 2019BERTScore: Evaluating Text Generation with BERTcited by 5,462
  10. 2022BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and Generationcited by 4,310
  11. 2016Improved Techniques for Training GANscited by 2,938
  12. 2022Imagen Video: High Definition Video Generation with Diffusion Modelscited by 1,765
35 more
  1. 2023ImageReward: Learning and Evaluating Human Preferences for Text-to-Image Generationcited by 1,010
  2. 2024Chameleon: Mixed-Modal Early-Fusion Foundation Modelscited by 904
  3. 2021BARTScore: Evaluating Generated Text as Text Generationcited by 719
  4. 2023DetectGPT: Zero-Shot Machine-Generated Text Detection using Probability Curvaturecited by 705
  5. 2018Texygen: A Benchmarking Platform for Text Generation Modelscited by 484
  6. 2021MAUVE: Measuring the Gap Between Neural Text and Human Text using Divergence Frontierscited by 466
  7. 2018HiDDeN: Hiding Data With Deep Networkscited by 384
  8. 2023DreamSim: Learning New Dimensions of Human Visual Similarity using Synthetic Datacited by 322
  9. 2022Language Models are Realistic Tabular Data Generatorscited by 290
  10. 2023Fast-DetectGPT: Efficient Zero-Shot Detection of Machine-Generated Text via Conditional Probability Curvaturecited by 278
  11. 2019StegaStamp: Invisible Hyperlinks in Physical Photographscited by 256
  12. 2024Spotting LLMs With Binoculars: Zero-Shot Detection of Machine-Generated Textcited by 255
  13. 2018DeepFakes: a New Threat to Face Recognition? Assessment and Detectioncited by 208
  14. 2023RADAR: Robust AI-Text Detection via Adversarial Learningcited by 187
  15. 2023GPT detectors are biased against non-native English writerscited by 175
  16. 2019Robust Invisible Video Watermarking with Attentioncited by 144
  17. 2021Watermarking Images in Self-Supervised Latent Spacescited by 90
  18. 2024WAVES: Benchmarking the Robustness of Image Watermarkscited by 89
  19. 2024RingID: Rethinking Tree-Ring Watermarking for Enhanced Multi-Key Identificationcited by 87
  20. 2022Machine Generated Text: A Comprehensive Survey of Threat Models and Detection Methodscited by 75
  21. 2023A Survey on Detection of LLMs-Generated Contentcited by 70
  22. 2021A survey of deep neural network watermarking techniquescited by 69
  23. 2024Proactive Detection of Voice Cloning with Localized Watermarkingcited by 67
  24. 2024Deepfake Generation and Detection: A Benchmark and Surveycited by 66
  25. 2023Red Teaming Language Model Detectors with Language Modelscited by 55
  26. 2023Domain Watermark: Effective and Harmless Dataset Copyright Protection is Closed at Handcited by 52
  27. 2020A Systematic Review on Model Watermarking for Neural Networkscited by 49
  28. 2021DVMark: A Deep Multiscale Framework for Video Watermarkingcited by 42
  29. 2023WavMark: Watermarking for Audio Generationcited by 41
  30. 2023Detecting Voice Cloning Attacks via Timbre Watermarkingcited by 39
  31. 2024V2A-Mark: Versatile Deep Visual-Audio Watermarking for Manipulation Localization and Copyright Protectioncited by 27
  32. 2024AudioMarkBench: Benchmarking Robustness of Audio Watermarkingcited by 19
  33. 2024Edit Distance Robust Watermarks via Indexing Pseudorandom Codescited by 12
  34. 2024Steganalysis on Digital Watermarking: Is Your Defense Truly Impervious?cited by 10
  35. 2024Securing the Future of GenAI: Policy and Technologycited by 4

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 29; 2026: 14.

43 papers.

Sort:

By area:

  1. 2025Are You Getting What You Pay For? Auditing Model Substitution in LLM APIscited by 35
  2. 2025SynthID-Image: Image watermarking at internet scalecited by 26
  3. 2025Frontier AI's Impact on the Cybersecurity Landscapecited by 21
  4. 2025Rethinking Data Protection in the (Generative) Artificial Intelligence Eracited by 17
  5. 2025SoK: Large Language Model Copyright Auditing via Fingerprintingcited by 14
  6. 2025PMark: Towards Robust and Distortion-free Semantic-level Watermarking with Channel Constraintscited by 14
  7. 2025Defending LLM Watermarking Against Spoofing Attacks with Contrastive Representation Learningcited by 10
  8. 2025Optimized Couplings for Watermarking Large Language Modelscited by 9
  9. 2025In-Context Watermarks for Large Language Modelscited by 9
  10. 2025Distributional Information Embedding: A Framework for Multi-bit Watermarkingcited by 8
  11. 2025DATABench: Evaluating Dataset Auditing in Deep Learning from an Adversarial Perspectivecited by 8
  12. 2025LLMs Have Rhythm: Fingerprinting Large Language Models Using Inter-Token Times and Network Traffic Analysiscited by 7
31 more
  1. 2025First-Place Solution to NeurIPS 2024 Invisible Watermark Removal Challengecited by 7
  2. 2025Position: LLM Watermarking Should Align Stakeholders' Incentives for Practical Adoptioncited by 7
  3. 2025Decoupling Content and Expression: Two-Dimensional Detection of AI-Generated Textcited by 6
  4. 2025WMCopier: Forging Invisible Image Watermarks on Arbitrary Imagescited by 5
  5. 2025Large Language Model Sourcing: A Surveycited by 5
  6. 2025A Crack in the Bark: Leveraging Public Knowledge to Remove Tree-Ring Watermarkscited by 4
  7. 2025Mitigating Watermark Forgery in Generative Models via Randomized Key Selectioncited by 4
  8. 2025Watermarking Degrades Alignment in Language Models: Analysis and Mitigationcited by 3
  9. 2026RAVEN: Erasing Invisible Watermarks via Novel View Synthesiscited by 2
  10. 2026Who Gets Flagged? The Pluralistic Evaluation Gap in AI Content Watermarkingcited by 2
  11. 2026Watermarking Should Be Treated as a Monitoring Primitivecited by 2
  12. 2025We Need Improved Data Curation and Attribution in AI for Scientific Discoverycited by 1
  13. 2025MetaSeal: Defending Against Image Attribution Forgery Through Content-Dependent Cryptographic Watermarkscited by 1
  14. 2025PVMark: Enabling Public Verifiability for LLM Watermarking Schemescited by 1
  15. 2025Performance Decay in Deepfake Detection: The Limitations of Training on Outdated Datacited by 1
  16. 2026SKeDA: A Generative Watermarking Framework for Text-to-video Diffusion Modelscited by 1
  17. 2025SOLIDO: A Robust Watermarking Method for Speech Synthesis via Low-Rank Adaptationcited by 0
  18. 2025Optimizing Token Choice for Code Watermarking: An RL Approachcited by 0
  19. 2025MOLM: Mixture of LoRA Markerscited by 0
  20. 2025Watermarking Large Language Models in Europe: Interpreting the AI Act in Light of Technologycited by 0
  21. 2025Adaptive Testing for Segmenting Watermarked Texts From Language Modelscited by 0
  22. 2026On Google's SynthID-Text LLM Watermarking System: Theoretical Analysis and Empirical Validationcited by 0
  23. 2026Towards Verifiable AI with Lightweight Cryptographic Proofs of Inferencecited by 0
  24. 2026Trustworthy AI: Ensuring Reliability and Accountability from Models to Agentscited by 0
  25. 2026"AI Watermarking": Bridging Policy Discourse and Technical Capabilitiescited by 0
  26. 2026Adversarial Attacks for Good: A Survey of Proactive Protection across the Visual Content Lifecyclecited by 0
  27. 2026Learning to Follow In-Context Watermark Instructions via Self-Distillationcited by 0
  28. 2026Asymmetric Phase Coding Video Watermarkingcited by 0
  29. 2026TANGO: Watermarking Masked Diffusion Language Models in Token Pairscited by 0
  30. 2026Exploring Weaknesses of Generative Image Watermarks against Latent Frequency Maskingcited by 0
  31. 2026Constitution-Guided Watermarkingcited by 0