SoK: Watermarking for AI-Generated Content
On the map: LLM safety and security → Watermarking generated content and models
Abstract (arXiv)
As the outputs of generative AI (GenAI) techniques improve in quality, it becomes increasingly challenging to distinguish them from human-created content. Watermarking schemes are a promising approach to address the problem of distinguishing between AI and human-generated content. These schemes embed hidden signals within AI-generated content to enable reliable detection. While watermarking is not a silver bullet for addressing all risks associated with GenAI, it can play a crucial role in enhancing AI safety and trustworthiness by combating misinformation and deception. This paper presents a comprehensive overview of watermarking techniques for GenAI, beginning with the need for watermarking from historical and regulatory perspectives. We formalize the definitions and desired properties of watermarking schemes and examine the key objectives and threat models for existing approaches. Practical evaluation strategies are also explored, providing insights into the development of robust watermarking techniques capable of resisting various attacks. Additionally, we review recent representative works, highlight open challenges, and discuss potential directions for this emerging field. By offering a thorough understanding of watermarking in GenAI, this work aims to guide researchers in advancing watermarking methods and applications, and support policymakers in addressing the broader implications of GenAI.
Key facts. Cited by 43 papers in the map (data through 7 Oct 2026). Placed in Watermarking generated content and models, one of 12 directions in LLM safety and security. Builds on 92 of its 147 references that are in the map. First posted on arXiv 27 Nov 2024.
Builds on
92 of 147 references.
Closest ancestors · 45
Same direction, newest first
- 2024An Undetectable Watermark for Generative Image Models9 Oct
- 2024New constructions of pseudorandom codes11 Sep
- 2024GROOT: Generating Robust Watermark for Diffusion-Model-Based Audio Synthesis15 Jul
- 2024Bileve: Securing Text Provenance in Large Language Models Against Spoofing with Bi-level Signature4 Jun
- 2024Watermarking Language Models for Many Adaptive Users17 May
- 2024MarkLLM: An Open-Source Toolkit for LLM Watermarking16 May
- 2024Gaussian Shading: Provable Performance-Lossless Image Watermarking for Diffusion Models7 Apr
- 2024A Statistical Framework of Watermarks for Large Language Models: Pivot, Detection Efficiency and Optimal Rules1 Apr
- 2024WaterMax: breaking the LLM watermark detectability-robustness-quality trade-off6 Mar
- 2024Watermark Stealing in Large Language Models29 Feb
- 2024Token-Specific Watermarking with Enhanced Detectability and Semantic Coherence for Large Language Models28 Feb
- 2024No Free Lunch in LLM Watermarking: Trade-offs in Watermarking Design Choices25 Feb
33 more
- 2024GumbelSoft: Diversified Language Model Watermarking via the GumbelMax-trick20 Feb
- 2024k-SemStamp: A Clustering-Based Semantic Watermark for Detection of Machine-Generated Text17 Feb
- 2024Proving membership in LLM pretraining data via data watermarks16 Feb
- 2024Pseudorandom Error-Correcting Codes14 Feb
- 2024Adaptive Text Watermark for Large Language Models25 Jan
- 2023Towards Optimal Statistical Watermarking13 Dec
- 2023A Survey of Text Watermarking in the Era of Large Language Models13 Dec
- 2023On the Learnability of Watermarks for Language Models7 Dec
- 2023Mark My Words: Analyzing and Evaluating Language Model Watermarks1 Dec
- 2023Watermarks in the Sand: Impossibility of Strong Watermarking for Generative Models7 Nov
- 2023Publicly-Detectable Watermarking for Language Models27 Oct
- 2023Embarrassingly Simple Text Watermarks13 Oct
- 2023A Resilient and Accessible Distribution-Preserving Watermark for Large Language Models11 Oct
- 2023Robustness of AI-Image Detectors: Fundamental Limits and Practical Attacks29 Sep
- 2023Unbiased Watermark for Large Language Models22 Sep
- 2023An Unforgeable Publicly Verifiable Watermark for Large Language Models30 Jul
- 2023Robust Distortion-free Watermarks for Language Models28 Jul
- 2023Provable Robust Watermarking for AI-Generated Text30 Jun
- 2023On the Reliability of Watermarks for Large Language Models7 Jun
- 2023Invisible Image Watermarks Are Provably Removable Using Generative AI2 Jun
- 2023Tree-Ring Watermarks: Fingerprints for Diffusion Images that are Invisible and Robust31 May
- 2023Undetectable Watermarks for Language Models25 May
- 2023Who Wrote this Code? Watermarking for Code Generation24 May
- 2023Evading Watermark based Detection of AI-Generated Content5 May
- 2023The Stable Signature: Rooting Watermarks in Latent Diffusion Models27 Mar
- 2023Paraphrasing evades detectors of AI-generated text, but retrieval is an effective defense23 Mar
- 2023A Recipe for Watermarking Diffusion Models17 Mar
- 2023Protecting Language Generation Models via Invisible Watermarking6 Feb
- 2023A Watermark for Large Language Models24 Jan
- 2022Distillation-Resistant Watermarking for Model Protection in NLP7 Oct
- 2022CATER: Intellectual Property Protection on Text Generation APIs via Conditional Watermarks19 Sep
- 2022The Ethical Need for Watermarks in Machine-Generated Language7 Sep
- 2022Attributable-Watermarking of Speech Generative Models17 Feb
Other references · 47
- 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
- 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
- 2020Denoising Diffusion Probabilistic Modelscited by 20,851
- 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
- 2018The Unreasonable Effectiveness of Deep Features as a Perceptual Metriccited by 11,029
- 2020Denoising Diffusion Implicit Modelscited by 9,655
- 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,393
- 2017Towards Deep Learning Models Resistant to Adversarial Attackscited by 6,326
- 2019BERTScore: Evaluating Text Generation with BERTcited by 5,462
- 2022BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and Generationcited by 4,310
- 2016Improved Techniques for Training GANscited by 2,938
- 2022Imagen Video: High Definition Video Generation with Diffusion Modelscited by 1,765
35 more
- 2023ImageReward: Learning and Evaluating Human Preferences for Text-to-Image Generationcited by 1,010
- 2024Chameleon: Mixed-Modal Early-Fusion Foundation Modelscited by 904
- 2021BARTScore: Evaluating Generated Text as Text Generationcited by 719
- 2023DetectGPT: Zero-Shot Machine-Generated Text Detection using Probability Curvaturecited by 705
- 2018Texygen: A Benchmarking Platform for Text Generation Modelscited by 484
- 2021MAUVE: Measuring the Gap Between Neural Text and Human Text using Divergence Frontierscited by 466
- 2018HiDDeN: Hiding Data With Deep Networkscited by 384
- 2023DreamSim: Learning New Dimensions of Human Visual Similarity using Synthetic Datacited by 322
- 2022Language Models are Realistic Tabular Data Generatorscited by 290
- 2023Fast-DetectGPT: Efficient Zero-Shot Detection of Machine-Generated Text via Conditional Probability Curvaturecited by 278
- 2019StegaStamp: Invisible Hyperlinks in Physical Photographscited by 256
- 2024Spotting LLMs With Binoculars: Zero-Shot Detection of Machine-Generated Textcited by 255
- 2018DeepFakes: a New Threat to Face Recognition? Assessment and Detectioncited by 208
- 2023RADAR: Robust AI-Text Detection via Adversarial Learningcited by 187
- 2023GPT detectors are biased against non-native English writerscited by 175
- 2019Robust Invisible Video Watermarking with Attentioncited by 144
- 2021Watermarking Images in Self-Supervised Latent Spacescited by 90
- 2024WAVES: Benchmarking the Robustness of Image Watermarkscited by 89
- 2024RingID: Rethinking Tree-Ring Watermarking for Enhanced Multi-Key Identificationcited by 87
- 2022Machine Generated Text: A Comprehensive Survey of Threat Models and Detection Methodscited by 75
- 2023A Survey on Detection of LLMs-Generated Contentcited by 70
- 2021A survey of deep neural network watermarking techniquescited by 69
- 2024Proactive Detection of Voice Cloning with Localized Watermarkingcited by 67
- 2024Deepfake Generation and Detection: A Benchmark and Surveycited by 66
- 2023Red Teaming Language Model Detectors with Language Modelscited by 55
- 2023Domain Watermark: Effective and Harmless Dataset Copyright Protection is Closed at Handcited by 52
- 2020A Systematic Review on Model Watermarking for Neural Networkscited by 49
- 2021DVMark: A Deep Multiscale Framework for Video Watermarkingcited by 42
- 2023WavMark: Watermarking for Audio Generationcited by 41
- 2023Detecting Voice Cloning Attacks via Timbre Watermarkingcited by 39
- 2024V2A-Mark: Versatile Deep Visual-Audio Watermarking for Manipulation Localization and Copyright Protectioncited by 27
- 2024AudioMarkBench: Benchmarking Robustness of Audio Watermarkingcited by 19
- 2024Edit Distance Robust Watermarks via Indexing Pseudorandom Codescited by 12
- 2024Steganalysis on Digital Watermarking: Is Your Defense Truly Impervious?cited by 10
- 2024Securing the Future of GenAI: Policy and Technologycited by 4
Built on by
Citing papers per year: 2025: 29; 2026: 14.
43 papers.
Sort:
By area:
- 2025Are You Getting What You Pay For? Auditing Model Substitution in LLM APIscited by 35
- 2025SynthID-Image: Image watermarking at internet scalecited by 26
- 2025Frontier AI's Impact on the Cybersecurity Landscapecited by 21
- 2025Rethinking Data Protection in the (Generative) Artificial Intelligence Eracited by 17
- 2025SoK: Large Language Model Copyright Auditing via Fingerprintingcited by 14
- 2025PMark: Towards Robust and Distortion-free Semantic-level Watermarking with Channel Constraintscited by 14
- 2025Defending LLM Watermarking Against Spoofing Attacks with Contrastive Representation Learningcited by 10
- 2025Optimized Couplings for Watermarking Large Language Modelscited by 9
- 2025In-Context Watermarks for Large Language Modelscited by 9
- 2025Distributional Information Embedding: A Framework for Multi-bit Watermarkingcited by 8
- 2025DATABench: Evaluating Dataset Auditing in Deep Learning from an Adversarial Perspectivecited by 8
- 2025LLMs Have Rhythm: Fingerprinting Large Language Models Using Inter-Token Times and Network Traffic Analysiscited by 7
31 more
- 2025First-Place Solution to NeurIPS 2024 Invisible Watermark Removal Challengecited by 7
- 2025Position: LLM Watermarking Should Align Stakeholders' Incentives for Practical Adoptioncited by 7
- 2025Decoupling Content and Expression: Two-Dimensional Detection of AI-Generated Textcited by 6
- 2025WMCopier: Forging Invisible Image Watermarks on Arbitrary Imagescited by 5
- 2025Large Language Model Sourcing: A Surveycited by 5
- 2025A Crack in the Bark: Leveraging Public Knowledge to Remove Tree-Ring Watermarkscited by 4
- 2025Mitigating Watermark Forgery in Generative Models via Randomized Key Selectioncited by 4
- 2025Watermarking Degrades Alignment in Language Models: Analysis and Mitigationcited by 3
- 2026RAVEN: Erasing Invisible Watermarks via Novel View Synthesiscited by 2
- 2026Who Gets Flagged? The Pluralistic Evaluation Gap in AI Content Watermarkingcited by 2
- 2026Watermarking Should Be Treated as a Monitoring Primitivecited by 2
- 2025We Need Improved Data Curation and Attribution in AI for Scientific Discoverycited by 1
- 2025MetaSeal: Defending Against Image Attribution Forgery Through Content-Dependent Cryptographic Watermarkscited by 1
- 2025PVMark: Enabling Public Verifiability for LLM Watermarking Schemescited by 1
- 2025Performance Decay in Deepfake Detection: The Limitations of Training on Outdated Datacited by 1
- 2026SKeDA: A Generative Watermarking Framework for Text-to-video Diffusion Modelscited by 1
- 2025SOLIDO: A Robust Watermarking Method for Speech Synthesis via Low-Rank Adaptationcited by 0
- 2025Optimizing Token Choice for Code Watermarking: An RL Approachcited by 0
- 2025MOLM: Mixture of LoRA Markerscited by 0
- 2025Watermarking Large Language Models in Europe: Interpreting the AI Act in Light of Technologycited by 0
- 2025Adaptive Testing for Segmenting Watermarked Texts From Language Modelscited by 0
- 2026On Google's SynthID-Text LLM Watermarking System: Theoretical Analysis and Empirical Validationcited by 0
- 2026Towards Verifiable AI with Lightweight Cryptographic Proofs of Inferencecited by 0
- 2026Trustworthy AI: Ensuring Reliability and Accountability from Models to Agentscited by 0
- 2026"AI Watermarking": Bridging Policy Discourse and Technical Capabilitiescited by 0
- 2026Adversarial Attacks for Good: A Survey of Proactive Protection across the Visual Content Lifecyclecited by 0
- 2026Learning to Follow In-Context Watermark Instructions via Self-Distillationcited by 0
- 2026Asymmetric Phase Coding Video Watermarkingcited by 0
- 2026TANGO: Watermarking Masked Diffusion Language Models in Token Pairscited by 0
- 2026Exploring Weaknesses of Generative Image Watermarks against Latent Frequency Maskingcited by 0
- 2026Constitution-Guided Watermarkingcited by 0
- 2026Constitution-Guided Watermarking7 Oct 2026
- 2026Exploring Weaknesses of Generative Image Watermarks against Latent Frequency Masking1 Oct 2026
- 2026TANGO: Watermarking Masked Diffusion Language Models in Token Pairs28 Sep 2026
- 2026Asymmetric Phase Coding Video Watermarking29 Aug 2026
- 2026Learning to Follow In-Context Watermark Instructions via Self-Distillation29 Aug 2026
- 2026Adversarial Attacks for Good: A Survey of Proactive Protection across the Visual Content Lifecycle5 Aug 2026
- 2026"AI Watermarking": Bridging Policy Discourse and Technical Capabilities25 May 2026
- 2026Watermarking Should Be Treated as a Monitoring Primitive13 May 2026
- 2026Trustworthy AI: Ensuring Reliability and Accountability from Models to Agents9 May 2026
- 2026Who Gets Flagged? The Pluralistic Evaluation Gap in AI Content Watermarking15 Apr 2026
- 2026Towards Verifiable AI with Lightweight Cryptographic Proofs of Inference19 Mar 2026
- 2026On Google's SynthID-Text LLM Watermarking System: Theoretical Analysis and Empirical Validation3 Mar 2026
31 more
- 2026SKeDA: A Generative Watermarking Framework for Text-to-video Diffusion Models27 Feb 2026
- 2026RAVEN: Erasing Invisible Watermarks via Novel View Synthesis13 Jan 2026
- 2025Performance Decay in Deepfake Detection: The Limitations of Training on Outdated Data10 Nov 2025
- 2025Adaptive Testing for Segmenting Watermarked Texts From Language Models10 Nov 2025
- 2025Watermarking Large Language Models in Europe: Interpreting the AI Act in Light of Technology5 Nov 2025
- 2025PVMark: Enabling Public Verifiability for LLM Watermarking Schemes30 Oct 2025
- 2025Position: LLM Watermarking Should Align Stakeholders' Incentives for Practical Adoption21 Oct 2025
- 2025Large Language Model Sourcing: A Survey11 Oct 2025
- 2025SynthID-Image: Image watermarking at internet scale10 Oct 2025
- 2025MOLM: Mixture of LoRA Markers30 Sep 2025
- 2025PMark: Towards Robust and Distortion-free Semantic-level Watermarking with Channel Constraints25 Sep 2025
- 2025MetaSeal: Defending Against Image Attribution Forgery Through Content-Dependent Cryptographic Watermarks13 Sep 2025
- 2025First-Place Solution to NeurIPS 2024 Invisible Watermark Removal Challenge28 Aug 2025
- 2025SoK: Large Language Model Copyright Auditing via Fingerprinting27 Aug 2025
- 2025Optimizing Token Choice for Code Watermarking: An RL Approach16 Aug 2025
- 2025Mitigating Watermark Forgery in Generative Models via Randomized Key Selection10 Jul 2025
- 2025DATABench: Evaluating Dataset Auditing in Deep Learning from an Adversarial Perspective8 Jul 2025
- 2025Rethinking Data Protection in the (Generative) Artificial Intelligence Era3 Jul 2025
- 2025A Crack in the Bark: Leveraging Public Knowledge to Remove Tree-Ring Watermarks12 Jun 2025
- 2025Watermarking Degrades Alignment in Language Models: Analysis and Mitigation4 Jun 2025
- 2025In-Context Watermarks for Large Language Models22 May 2025
- 2025Optimized Couplings for Watermarking Large Language Models13 May 2025
- 2025SOLIDO: A Robust Watermarking Method for Speech Synthesis via Low-Rank Adaptation21 Apr 2025
- 2025Defending LLM Watermarking Against Spoofing Attacks with Contrastive Representation Learning9 Apr 2025
- 2025Frontier AI's Impact on the Cybersecurity Landscape7 Apr 2025
- 2025Are You Getting What You Pay For? Auditing Model Substitution in LLM APIs7 Apr 2025
- 2025We Need Improved Data Curation and Attribution in AI for Scientific Discovery3 Apr 2025
- 2025WMCopier: Forging Invisible Image Watermarks on Arbitrary Images28 Mar 2025
- 2025Decoupling Content and Expression: Two-Dimensional Detection of AI-Generated Text1 Mar 2025
- 2025LLMs Have Rhythm: Fingerprinting Large Language Models Using Inter-Token Times and Network Traffic Analysis27 Feb 2025
- 2025Distributional Information Embedding: A Framework for Multi-bit Watermarking27 Jan 2025
- 2025PMark: Towards Robust and Distortion-free Semantic-level Watermarking with Channel Constraintscited by 14
- 2025Defending LLM Watermarking Against Spoofing Attacks with Contrastive Representation Learningcited by 10
- 2025Optimized Couplings for Watermarking Large Language Modelscited by 9
- 2025In-Context Watermarks for Large Language Modelscited by 9
- 2025Position: LLM Watermarking Should Align Stakeholders' Incentives for Practical Adoptioncited by 7
- 2025Mitigating Watermark Forgery in Generative Models via Randomized Key Selectioncited by 4
- 2026Who Gets Flagged? The Pluralistic Evaluation Gap in AI Content Watermarkingcited by 2
- 2025We Need Improved Data Curation and Attribution in AI for Scientific Discoverycited by 1
- 2025PVMark: Enabling Public Verifiability for LLM Watermarking Schemescited by 1
- 2025Optimizing Token Choice for Code Watermarking: An RL Approachcited by 0
- 2025MOLM: Mixture of LoRA Markerscited by 0
- 2025Watermarking Large Language Models in Europe: Interpreting the AI Act in Light of Technologycited by 0
12 most cited of 17 All 17 (JSON)
- 2026Exploring Weaknesses of Generative Image Watermarks against Latent Frequency Masking1 Oct 2026
- 2026Learning to Follow In-Context Watermark Instructions via Self-Distillation29 Aug 2026
- 2026"AI Watermarking": Bridging Policy Discourse and Technical Capabilities25 May 2026
- 2026Who Gets Flagged? The Pluralistic Evaluation Gap in AI Content Watermarking15 Apr 2026
- 2026On Google's SynthID-Text LLM Watermarking System: Theoretical Analysis and Empirical Validation3 Mar 2026
- 2025Adaptive Testing for Segmenting Watermarked Texts From Language Models10 Nov 2025
- 2025Watermarking Large Language Models in Europe: Interpreting the AI Act in Light of Technology5 Nov 2025
- 2025PVMark: Enabling Public Verifiability for LLM Watermarking Schemes30 Oct 2025
- 2025Position: LLM Watermarking Should Align Stakeholders' Incentives for Practical Adoption21 Oct 2025
- 2025MOLM: Mixture of LoRA Markers30 Sep 2025
- 2025PMark: Towards Robust and Distortion-free Semantic-level Watermarking with Channel Constraints25 Sep 2025
- 2025Optimizing Token Choice for Code Watermarking: An RL Approach16 Aug 2025
12 newest of 17 All 17 (JSON)
- 2025A Crack in the Bark: Leveraging Public Knowledge to Remove Tree-Ring Watermarkscited by 4
- 2026SKeDA: A Generative Watermarking Framework for Text-to-video Diffusion Modelscited by 1
- 2025SOLIDO: A Robust Watermarking Method for Speech Synthesis via Low-Rank Adaptationcited by 0
- 2026TANGO: Watermarking Masked Diffusion Language Models in Token Pairscited by 0
- 2026TANGO: Watermarking Masked Diffusion Language Models in Token Pairs28 Sep 2026
- 2026SKeDA: A Generative Watermarking Framework for Text-to-video Diffusion Models27 Feb 2026
- 2025A Crack in the Bark: Leveraging Public Knowledge to Remove Tree-Ring Watermarks12 Jun 2025
- 2025SOLIDO: A Robust Watermarking Method for Speech Synthesis via Low-Rank Adaptation21 Apr 2025
- 2025Are You Getting What You Pay For? Auditing Model Substitution in LLM APIscited by 35
- 2025SynthID-Image: Image watermarking at internet scalecited by 26
- 2025Frontier AI's Impact on the Cybersecurity Landscapecited by 21
- 2025Rethinking Data Protection in the (Generative) Artificial Intelligence Eracited by 17
- 2025SoK: Large Language Model Copyright Auditing via Fingerprintingcited by 14
- 2025Distributional Information Embedding: A Framework for Multi-bit Watermarkingcited by 8
- 2025DATABench: Evaluating Dataset Auditing in Deep Learning from an Adversarial Perspectivecited by 8
- 2025LLMs Have Rhythm: Fingerprinting Large Language Models Using Inter-Token Times and Network Traffic Analysiscited by 7
- 2025First-Place Solution to NeurIPS 2024 Invisible Watermark Removal Challengecited by 7
- 2025Decoupling Content and Expression: Two-Dimensional Detection of AI-Generated Textcited by 6
- 2025WMCopier: Forging Invisible Image Watermarks on Arbitrary Imagescited by 5
- 2025Large Language Model Sourcing: A Surveycited by 5
12 most cited of 20 All 20 (JSON)
- 2026Constitution-Guided Watermarking7 Oct 2026
- 2026Asymmetric Phase Coding Video Watermarking29 Aug 2026
- 2026Adversarial Attacks for Good: A Survey of Proactive Protection across the Visual Content Lifecycle5 Aug 2026
- 2026Watermarking Should Be Treated as a Monitoring Primitive13 May 2026
- 2026Towards Verifiable AI with Lightweight Cryptographic Proofs of Inference19 Mar 2026
- 2026RAVEN: Erasing Invisible Watermarks via Novel View Synthesis13 Jan 2026
- 2025Performance Decay in Deepfake Detection: The Limitations of Training on Outdated Data10 Nov 2025
- 2025Large Language Model Sourcing: A Survey11 Oct 2025
- 2025SynthID-Image: Image watermarking at internet scale10 Oct 2025
- 2025MetaSeal: Defending Against Image Attribution Forgery Through Content-Dependent Cryptographic Watermarks13 Sep 2025
- 2025First-Place Solution to NeurIPS 2024 Invisible Watermark Removal Challenge28 Aug 2025
- 2025SoK: Large Language Model Copyright Auditing via Fingerprinting27 Aug 2025
12 newest of 20 All 20 (JSON)