# SoK: Watermarking for AI-Generated Content

arXiv: https://arxiv.org/abs/2411.18479 · HTML: https://mlsanity.com/paper/2411.18479/

## Key facts

- Cited by 51 papers in the map (data through 14 Oct 2026).
- Placed in Watermarking generated content and models, one of 12 directions in LLM safety and security.
- Builds on 92 of its 147 references that are in the map.
- First posted on arXiv 27 Nov 2024.

## Placed in

- LLM safety and security / [Watermarking generated content and models](https://mlsanity.com/safety/watermarking-generated-content-and-models/)

## Builds on (most cited first, 30 of 92)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [High-Resolution Image Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2112.10752/) (2021)
- [The Unreasonable Effectiveness of Deep Features as a Perceptual Metric](https://mlsanity.com/paper/1801.03924/) (2018)
- [Denoising Diffusion Implicit Models](https://mlsanity.com/paper/2010.02502/) (2020)
- [Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena](https://mlsanity.com/paper/2306.05685/) (2023)
- [Towards Deep Learning Models Resistant to Adversarial Attacks](https://mlsanity.com/paper/1706.06083/) (2017)
- [BERTScore: Evaluating Text Generation with BERT](https://mlsanity.com/paper/1904.09675/) (2019)
- [BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and Generation](https://mlsanity.com/paper/2201.12086/) (2022)
- [Improved Techniques for Training GANs](https://mlsanity.com/paper/1606.03498/) (2016)
- [Imagen Video: High Definition Video Generation with Diffusion Models](https://mlsanity.com/paper/2210.02303/) (2022)
- [ImageReward: Learning and Evaluating Human Preferences for Text-to-Image Generation](https://mlsanity.com/paper/2304.05977/) (2023)
- [Chameleon: Mixed-Modal Early-Fusion Foundation Models](https://mlsanity.com/paper/2405.09818/) (2024)
- [DetectGPT: Zero-Shot Machine-Generated Text Detection using Probability Curvature](https://mlsanity.com/paper/2301.11305/) (2023)
- [A Watermark for Large Language Models](https://mlsanity.com/paper/2301.10226/) (2023)
- [BARTScore: Evaluating Generated Text as Text Generation](https://mlsanity.com/paper/2106.11520/) (2021)
- [Texygen: A Benchmarking Platform for Text Generation Models](https://mlsanity.com/paper/1802.01886/) (2018)
- [MAUVE: Measuring the Gap Between Neural Text and Human Text using Divergence Frontiers](https://mlsanity.com/paper/2102.01454/) (2021)
- [Paraphrasing evades detectors of AI-generated text, but retrieval is an effective defense](https://mlsanity.com/paper/2303.13408/) (2023)
- [HiDDeN: Hiding Data With Deep Networks](https://mlsanity.com/paper/1807.09937/) (2018)
- [DreamSim: Learning New Dimensions of Human Visual Similarity using Synthetic Data](https://mlsanity.com/paper/2306.09344/) (2023)
- [Robust Distortion-free Watermarks for Language Models](https://mlsanity.com/paper/2307.15593/) (2023)
- [The Stable Signature: Rooting Watermarks in Latent Diffusion Models](https://mlsanity.com/paper/2303.15435/) (2023)
- [Provable Robust Watermarking for AI-Generated Text](https://mlsanity.com/paper/2306.17439/) (2023)
- [Language Models are Realistic Tabular Data Generators](https://mlsanity.com/paper/2210.06280/) (2022)
- [Fast-DetectGPT: Efficient Zero-Shot Detection of Machine-Generated Text via Conditional Probability Curvature](https://mlsanity.com/paper/2310.05130/) (2023)
- [StegaStamp: Invisible Hyperlinks in Physical Photographs](https://mlsanity.com/paper/1904.05343/) (2019)
- [Spotting LLMs With Binoculars: Zero-Shot Detection of Machine-Generated Text](https://mlsanity.com/paper/2401.12070/) (2024)
- [Undetectable Watermarks for Language Models](https://mlsanity.com/paper/2306.09194/) (2023)

## Built on by (most cited first, 30 of 51)

- [Are You Getting What You Pay For? Auditing Model Substitution in LLM APIs](https://mlsanity.com/paper/2504.04715/) (2025)
- [SynthID-Image: Image watermarking at internet scale](https://mlsanity.com/paper/2510.09263/) (2025)
- [Frontier AI's Impact on the Cybersecurity Landscape](https://mlsanity.com/paper/2504.05408/) (2025)
- [Rethinking Data Protection in the (Generative) Artificial Intelligence Era](https://mlsanity.com/paper/2507.03034/) (2025)
- [PMark: Towards Robust and Distortion-free Semantic-level Watermarking with Channel Constraints](https://mlsanity.com/paper/2509.21057/) (2025)
- [SoK: Large Language Model Copyright Auditing via Fingerprinting](https://mlsanity.com/paper/2508.19843/) (2025)
- [Defending LLM Watermarking Against Spoofing Attacks with Contrastive Representation Learning](https://mlsanity.com/paper/2504.06575/) (2025)
- [In-Context Watermarks for Large Language Models](https://mlsanity.com/paper/2505.16934/) (2025)
- [Distributional Information Embedding: A Framework for Multi-bit Watermarking](https://mlsanity.com/paper/2501.16558/) (2025)
- [Optimized Couplings for Watermarking Large Language Models](https://mlsanity.com/paper/2505.08878/) (2025)
- [Watermarking Without Standards Is Not AI Governance](https://mlsanity.com/paper/2505.23814/) (2025)
- [First-Place Solution to NeurIPS 2024 Invisible Watermark Removal Challenge](https://mlsanity.com/paper/2508.21072/) (2025)
- [DATABench: Evaluating Dataset Auditing in Deep Learning from an Adversarial Perspective](https://mlsanity.com/paper/2507.05622/) (2025)
- [LLMs Have Rhythm: Fingerprinting Large Language Models Using Inter-Token Times and Network Traffic Analysis](https://mlsanity.com/paper/2502.20589/) (2025)
- [Position: LLM Watermarking Should Align Stakeholders' Incentives for Practical Adoption](https://mlsanity.com/paper/2510.18333/) (2025)
- [Decoupling Content and Expression: Two-Dimensional Detection of AI-Generated Text](https://mlsanity.com/paper/2503.00258/) (2025)
- [WMCopier: Forging Invisible Image Watermarks on Arbitrary Images](https://mlsanity.com/paper/2503.22330/) (2025)
- [Large Language Model Sourcing: A Survey](https://mlsanity.com/paper/2510.10161/) (2025)
- [A Crack in the Bark: Leveraging Public Knowledge to Remove Tree-Ring Watermarks](https://mlsanity.com/paper/2506.10502/) (2025)
- [Mitigating Watermark Forgery in Generative Models via Randomized Key Selection](https://mlsanity.com/paper/2507.07871/) (2025)
- [MarkSweep: A No-box Removal Attack on AI-Generated Image Watermarking via Noise Intensification and Frequency-aware Denoising](https://mlsanity.com/paper/2602.15364/) (2026)
- [Watermarking for AI Content Detection: A Review on Text, Visual, and Audio Modalities](https://mlsanity.com/paper/2504.03765/) (2025)
- [Watermarking Degrades Alignment in Language Models: Analysis and Mitigation](https://mlsanity.com/paper/2506.04462/) (2025)
- [PVMark: Enabling Public Verifiability for LLM Watermarking Schemes](https://mlsanity.com/paper/2510.26274/) (2025)
- [RAVEN: Erasing Invisible Watermarks via Novel View Synthesis](https://mlsanity.com/paper/2601.08832/) (2026)
- [Who Gets Flagged? The Pluralistic Evaluation Gap in AI Content Watermarking](https://mlsanity.com/paper/2604.13776/) (2026)
- [Watermarking Should Be Treated as a Monitoring Primitive](https://mlsanity.com/paper/2605.13095/) (2026)
- [Adversarial Attacks for Good: A Survey of Proactive Protection across the Visual Content Lifecycle](https://mlsanity.com/paper/2608.04314/) (2026)
- [We Need Improved Data Curation and Attribution in AI for Scientific Discovery](https://mlsanity.com/paper/2504.02486/) (2025)
- [SOLIDO: A Robust Watermarking Method for Speech Synthesis via Low-Rank Adaptation](https://mlsanity.com/paper/2504.15035/) (2025)

## Abstract (arXiv)

As the outputs of generative AI (GenAI) techniques improve in quality, it becomes increasingly challenging to distinguish them from human-created content. Watermarking schemes are a promising approach to address the problem of distinguishing between AI and human-generated content. These schemes embed hidden signals within AI-generated content to enable reliable detection. While watermarking is not a silver bullet for addressing all risks associated with GenAI, it can play a crucial role in enhancing AI safety and trustworthiness by combating misinformation and deception. This paper presents a comprehensive overview of watermarking techniques for GenAI, beginning with the need for watermarking from historical and regulatory perspectives. We formalize the definitions and desired properties of watermarking schemes and examine the key objectives and threat models for existing approaches. Practical evaluation strategies are also explored, providing insights into the development of robust watermarking techniques capable of resisting various attacks. Additionally, we review recent representative works, highlight open challenges, and discuss potential directions for this emerging field. By offering a thorough understanding of watermarking in GenAI, this work aims to guide researchers in advancing watermarking methods and applications, and support policymakers in addressing the broader implications of GenAI.