mlsanityData to 14 Oct 2026

A Watermark for Large Language Models

arXiv 2301.10226 ↗First posted 24 Jan 2023cs.LG cs.CL cs.CRcited by 725

On the map: LLM safety and security → Watermarking generated content and models

Abstract (arXiv)

Potential harms of large language models can be mitigated by watermarking model output, i.e., embedding signals into generated text that are invisible to humans but algorithmically detectable from a short span of tokens. We propose a watermarking framework for proprietary language models. The watermark can be embedded with negligible impact on text quality, and can be detected using an efficient open-source algorithm without access to the language model API or parameters. The watermark works by selecting a randomized set of "green" tokens before a word is generated, and then softly promoting use of green tokens during sampling. We propose a statistical test for detecting the watermark with interpretable p-values, and derive an information-theoretic framework for analyzing the sensitivity of the watermark. We test the watermark using a multi-billion parameter model from the Open Pretrained Transformer (OPT) family, and discuss robustness and security.

Key facts. Cited by 725 papers in the map (data through 14 Oct 2026). Placed in Watermarking generated content and models, one of 12 directions in LLM safety and security. Builds on 29 of its 64 references that are in the map. First posted on arXiv 24 Jan 2023.

Builds on

29 of 64 references.

Closest ancestors · 4

Same direction, newest first

  1. 2022Watermarking Pre-trained Language Models with Backdooring14 Oct
  2. 2022CATER: Intellectual Property Protection on Text Generation APIs via Conditional Watermarks19 Sep
  3. 2022The Ethical Need for Watermarks in Machine-Generated Language7 Sep
  4. 2021Protecting Intellectual Property of Language Generation APIs with Lexical Watermark5 Dec

Other references · 25

  1. 2017Attention Is All You Needcited by 60,361
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  3. 2022Training language models to follow instructions with human feedbackcited by 15,971
  4. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,599
  5. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,372
  6. 2022Robust Speech Recognition via Large-Scale Weak Supervisioncited by 4,057
  7. 2017Deep reinforcement learning from human preferencescited by 3,912
  8. 2017TriviaQA: A Large Scale Distantly Supervised Challenge Dataset for Reading Comprehensioncited by 2,955
  9. 2019Defending Against Neural Fake Newscited by 559
  10. 2018HiDDeN: Hiding Data With Deep Networkscited by 397
  11. 2018Turning Your Weakness Into a Strength: Watermarking Deep Neural Networks by Backdooringcited by 357
  12. 2020Automatic Detection of Machine Generated Text: A Critical Surveycited by 166
13 more
  1. 2020Adversarial Watermarking Transformer: Towards Tracing Text Provenance with Data Hidingcited by 130
  2. 2022Machine Generated Text: A Comprehensive Survey of Threat Models and Detection Methodscited by 79
  3. 2019Neural Linguistic Steganographycited by 78
  4. 2021Bad Characters: Imperceptible NLP Attackscited by 71
  5. 2020A Systematic Review on Model Watermarking for Neural Networkscited by 52
  6. 2017Generating Steganographic Text with LSTMscited by 44
  7. 2020Detecting Cross-Modal Inconsistency to Defend Against Neural Fake Newscited by 42
  8. 2020Attacking Neural Text Detectorscited by 39
  9. 2021The Threat of Offensive AI to Organizationscited by 38
  10. 2019Towards Near-imperceptible Steganographic Textcited by 32
  11. 2021Frustratingly Easy Edit-based Linguistic Steganography with a Masked Language Modelcited by 30
  12. 2020Reverse Engineering Configurations of Neural Text Generation Modelscited by 15
  13. 2021Fall of Giants: How popular text-based MLaaS fall against a simple evasion attackcited by 13

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 140; 2024: 172; 2025: 192; 2026: 221.

725 papers.

Sort:

By area:

  1. 2023DetectGPT: Zero-Shot Machine-Generated Text Detection using Probability Curvaturecited by 726
  2. 2024Moshi: a speech-text foundation model for real-time dialoguecited by 568
  3. 2023Paraphrasing evades detectors of AI-generated text, but retrieval is an effective defensecited by 429
  4. 2023Trustworthy LLMs: a Survey and Guideline for Evaluating Large Language Models' Alignmentcited by 401
  5. 2023Can AI-Generated Text be Reliably Detected?cited by 386
  6. 2023Provable Robust Watermarking for AI-Generated Textcited by 299
  7. 2023Fast-DetectGPT: Efficient Zero-Shot Detection of Machine-Generated Text via Conditional Probability Curvaturecited by 287
  8. 2024Spotting LLMs With Binoculars: Zero-Shot Detection of Machine-Generated Textcited by 263
  9. 2023Challenges and Applications of Large Language Modelscited by 254
  10. 2023Self-Consuming Generative Models Go MADcited by 250
  11. 2023SugarCrepe: Fixing Hackable Benchmarks for Vision-Language Compositionalitycited by 243
  12. 2023AI Control: Improving Safety Despite Intentional Subversioncited by 229
38 more
  1. 2023Undetectable Watermarks for Language Modelscited by 226
  2. 2023DetectLLM: Leveraging Log Rank Information for Zero-Shot Detection of Machine-Generated Textcited by 210
  3. 2025Large Language Model Agent: A Survey on Methodology, Applications and Challengescited by 205
  4. 2023RADAR: Robust AI-Text Detection via Adversarial Learningcited by 190
  5. 2023On the Reliability of Watermarks for Large Language Modelscited by 189
  6. 2023AI Deception: A Survey of Examples, Risks, and Potential Solutionscited by 188
  7. 2023Tree-Ring Watermarks: Fingerprints for Diffusion Images that are Invisible and Robustcited by 187
  8. 2023GPT detectors are biased against non-native English writerscited by 178
  9. 2023Can LLM-Generated Misinformation Be Detected?cited by 177
  10. 2023A Categorical Archive of ChatGPT Failurescited by 168
  11. 2024Monitoring AI-Modified Content at Scale: A Case Study on the Impact of ChatGPT on AI Conference Peer Reviewscited by 164
  12. 2023Raising the Cost of Malicious AI-Powered Image Editingcited by 163
  13. 2023Sociotechnical Safety Evaluation of Generative AI Systemscited by 157
  14. 2023Stop Uploading Test Data in Plain Text: Practical Strategies for Mitigating Data Contamination by Evaluation Benchmarkscited by 142
  15. 2023A Recipe for Watermarking Diffusion Modelscited by 141
  16. 2023Who Wrote this Code? Watermarking for Code Generationcited by 141
  17. 2023Auditing large language models: a three-layered approachcited by 137
  18. 2023DNA-GPT: Divergent N-Gram Analysis for Training-Free Detection of GPT-Generated Textcited by 129
  19. 2023The Science of Detecting LLM-Generated Textscited by 128
  20. 2023Explainable Artificial Intelligence (XAI) 2.0: A Manifesto of Open Challenges and Interdisciplinary Research Directionscited by 125
  21. 2023AI Transparency in the Age of LLMs: A Human-Centered Research Roadmapcited by 123
  22. 2023M4: Multi-generator, Multi-domain, and Multi-lingual Black-Box Machine-Generated Text Detectioncited by 121
  23. 2024Mapping the Increasing Use of LLMs in Scientific Paperscited by 118
  24. 2023Intrinsic Dimension Estimation for Robust Detection of AI-Generated Textscited by 112
  25. 2023A Semantic Invariant Robust Watermark for Large Language Modelscited by 112
  26. 2023Unbiased Watermark for Large Language Modelscited by 111
  27. 2023AutoDAN: Interpretable Gradient-Based Adversarial Attacks on Large Language Modelscited by 111
  28. 2023A Survey on LLM-Generated Text Detection: Necessity, Methods, and Future Directionscited by 105
  29. 2023Regulating ChatGPT and other Large Generative AI Modelscited by 104
  30. 2023Artificial Artificial Artificial Intelligence: Crowd Workers Widely Use Large Language Models for Text Production Taskscited by 103
  31. 2024MarkLLM: An Open-Source Toolkit for LLM Watermarkingcited by 95
  32. 2025Subliminal Learning: Language models transmit behavioral traits via hidden signals in datacited by 95
  33. 2023Language Model Behavior: A Comprehensive Surveycited by 92
  34. 2023Evaluating the Social Impact of Generative AI Systems in Systems and Societycited by 92
  35. 2024Watermark Stealing in Large Language Modelscited by 92
  36. 2023Ghostbuster: Detecting Text Ghostwritten by Large Language Modelscited by 91
  37. 2023Identifying and Mitigating the Security Risks of Generative AIcited by 91
  38. 2023Red teaming ChatGPT via Jailbreaking: Bias, Robustness, Reliability and Toxicitycited by 89

50 most cited of 725 All 725 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2023Robust Distortion-free Watermarks for Language Models
  2. 2024A Statistical Framework of Watermarks for Large Language Models: Pivot, Detection Efficiency and Optimal Rules
  3. 2024PostMark: A Robust Blackbox Watermark for Large Language Models
  4. 2025GaussMark: A Practical Approach for Structural Watermarking of Language Models
  5. 2025Improved Unbiased Watermark for Large Language Models
  6. 2025Security and Detectability Analysis of Unicode Text Watermarking Methods against Large Language Models
  7. 2023Improving the Generation Quality of Watermarked Large Language Models via Word Importance Scoring
  8. 2025MorphMark: Flexible Adaptive Watermarking for Large Language Models