mlsanityData to 7 Oct 2026

No Free Lunch in LLM Watermarking: Trade-offs in Watermarking Design Choices

arXiv 2402.16187 ↗First posted 25 Feb 2024cs.CR cs.CL cs.LGcited by 47

On the map: LLM safety and security → Watermarking generated content and models

Abstract (arXiv)

Advances in generative models have made it possible for AI-generated text, code, and images to mirror human-generated content in many applications. Watermarking, a technique that aims to embed information in the output of a model to verify its source, is useful for mitigating the misuse of such AI-generated content. However, we show that common design choices in LLM watermarking schemes make the resulting systems surprisingly susceptible to attack -- leading to fundamental trade-offs in robustness, utility, and usability. To navigate these trade-offs, we rigorously study a set of simple yet effective attacks on common watermarking systems, and propose guidelines and defenses for LLM watermarking in practice.

Key facts. Cited by 47 papers in the map (data through 7 Oct 2026). Placed in Watermarking generated content and models, one of 12 directions in LLM safety and security. Builds on 27 of its 41 references that are in the map. First posted on arXiv 25 Feb 2024.

Builds on

27 of 41 references.

Closest ancestors · 15

Same direction, newest first

  1. 2023On the Learnability of Watermarks for Language Models7 Dec
  2. 2023Mark My Words: Analyzing and Evaluating Language Model Watermarks1 Dec
  3. 2023A Robust Semantics-based Watermark for Large Language Model against Paraphrasing15 Nov
  4. 2023Watermarks in the Sand: Impossibility of Strong Watermarking for Generative Models7 Nov
  5. 2023Publicly-Detectable Watermarking for Language Models27 Oct
  6. 2023A Semantic Invariant Robust Watermark for Large Language Models10 Oct
  7. 2023Unbiased Watermark for Large Language Models22 Sep
  8. 2023Robust Distortion-free Watermarks for Language Models28 Jul
  9. 2023Provable Robust Watermarking for AI-Generated Text30 Jun
  10. 2023On the Reliability of Watermarks for Large Language Models7 Jun
  11. 2023Tree-Ring Watermarks: Fingerprints for Diffusion Images that are Invisible and Robust31 May
  12. 2023Undetectable Watermarks for Language Models25 May
3 more
  1. 2023Paraphrasing evades detectors of AI-generated text, but retrieval is an effective defense23 Mar
  2. 2023Can AI-Generated Text be Reliably Detected?17 Mar
  3. 2023A Watermark for Large Language Models24 Jan

Other references · 12

  1. 2020Language Models are Few-Shot Learnerscited by 32,199
  2. 2023GPT-4 Technical Reportcited by 18,132
  3. 2022Training language models to follow instructions with human feedbackcited by 15,712
  4. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,865
  5. 2022Photorealistic Text-to-Image Diffusion Models with Deep Language Understandingcited by 6,322
  6. 2022OPT: Open Pre-trained Transformer Language Modelscited by 3,464
  7. 2023DetectGPT: Zero-Shot Machine-Generated Text Detection using Probability Curvaturecited by 705
  8. 2019Release Strategies and the Social Impacts of Language Modelscited by 505
  9. 2018Adversarial Example Generation with Syntactically Controlled Paraphrase Networkscited by 282
  10. 2017Paraphrase Generation with Deep Reinforcement Learningcited by 58
  11. 2023Stealing the Decoding Algorithms of Language Modelscited by 19
  12. 2021Towards Document-Level Paraphrase Generation with Sentence Rewriting and Reorderingcited by 10

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 7; 2025: 20; 2026: 20.

47 papers.

Sort:

By area:

  1. 2024SoK: Watermarking for AI-Generated Contentcited by 43
  2. 2024Bileve: Securing Text Provenance in Large Language Models Against Spoofing with Bi-level Signaturecited by 21
  3. 2026Large Language Model Reasoning Failurescited by 20
  4. 2025BiMark: Unbiased Multilayer Watermarking for Large Language Modelscited by 18
  5. 2024Beyond Binary: Towards Fine-Grained LLM-Generated Text Detection via Role Recognition and Involvement Measurementcited by 16
  6. 2024Optimizing Adaptive Attacks against Watermarks for Language Modelscited by 11
  7. 2025Defending LLM Watermarking Against Spoofing Attacks with Contrastive Representation Learningcited by 10
  8. 2025Enhancing LLM Watermark Resilience Against Both Scrubbing and Spoofing Attackscited by 8
  9. 2025Marking Code Without Breaking It: Code Watermarking for Detecting LLM-Generated Codecited by 7
  10. 2025Character-Level Perturbations Disrupt LLM Watermarkscited by 7
  11. 2025Position: LLM Watermarking Should Align Stakeholders' Incentives for Practical Adoptioncited by 7
  12. 2024Enhancing Training Data Attribution for Large Language Models with Fitting Error Considerationcited by 6
35 more
  1. 2025From Trade-off to Synergy: A Versatile Symbiotic Watermarking Framework for Large Language Modelscited by 5
  2. 2024Building Intelligence Identification System via Large Language Model Watermarking: A Survey and Beyondcited by 4
  3. 2025Evaluating the Robustness and Accuracy of Text Watermarking Under Real-World Cross-Lingual Manipulationscited by 4
  4. 2025Mitigating Watermark Forgery in Generative Models via Randomized Key Selectioncited by 4
  5. 2025DITTO: A Spoofing Attack Framework on Watermarked LLMs via Knowledge Distillationcited by 4
  6. 2025WaterMod: Modular Token-Rank Partitioning for Probability-Balanced LLM Watermarkingcited by 4
  7. 2025Beyond Easy Wins: A Text Hardness-Aware Benchmark for LLM-generated Text Detectioncited by 3
  8. 2025Detecting Post-generation Edits to Watermarked LLM Outputs via Combinatorial Watermarkingcited by 3
  9. 2026Stateless Yet Not Forgetful: Implicit Memory as a Hidden Channel in LLMscited by 3
  10. 2026Watermarking LLM Agent Trajectoriescited by 3
  11. 2026Sequential Behavioral Watermarking for LLM Agentscited by 3
  12. 2025Sandcastles in the Storm: Revisiting the (Im)possibility of Strong Watermarkingcited by 2
  13. 2025PhantomHunter: Detecting Unseen Privately-Tuned LLM-Generated Text via Family-Aware Learningcited by 2
  14. 2026Watermarking Should Be Treated as a Monitoring Primitivecited by 2
  15. 2024CLUE-MARK: Watermarking Diffusion Models using CLWEcited by 1
  16. 2025PVMark: Enabling Public Verifiability for LLM Watermarking Schemescited by 1
  17. 2026Robust LLM Watermarking with Minimal Semantic Distortion for IP Protectioncited by 1
  18. 2026Tracing Provenance and Detecting Tampering with Complementary LLM Watermarkscited by 1
  19. 2025Modification and Generated-Text Detection: Achieving Dual Detection Capabilities for the Outputs of LLM by Watermarkcited by 0
  20. 2025Breaking Distortion-free Watermarks in Large Language Modelscited by 0
  21. 2025An LLM Framework For Cryptography Over Chat Channelscited by 0
  22. 2025DualGuard: Dual-stream Large Language Model Watermarking Defense against Paraphrase and Spoofing Attackcited by 0
  23. 2026Proof-of-Authorship for Diffusion-based AI Generated Contentcited by 0
  24. 2026From AI-Generated Content to Agentic Action: Security and Safety Threats in Generative AIcited by 0
  25. 2026Blind PRNG Hijacking: An Undetectable Integrity-Preserving Attack Against LLM Watermarkingcited by 0
  26. 2026Linear Ensembles Wash Away Watermarks: On the Fragility of Distributional Perturbations in LLMscited by 0
  27. 2026Position: Preventing AI-Generated CSAM Necessitates New Approaches to AI Safetycited by 0
  28. 2026LLM Watermarking as Big Data Provenance: A Deployment-Oriented Systematizationcited by 0
  29. 2026Making Open-Source Text LLM Watermarks Durable Against Mergingcited by 0
  30. 2026Linguistic Holonomy and Statistical Watermarks: Inner Geometry of Meaning-Preserving Transformationscited by 0
  31. 2026Semantic Watermarking with Order-Robust Detection over Sub-sentence Unitscited by 0
  32. 2026Minimax bounds for watermarked and masked recursive discrete distribution estimationcited by 0
  33. 2026Codetta: High-Capacity, Keyless, and Undetectable Multi-Agent Collusioncited by 0
  34. 2026GenoTrace: Inheritable Watermarks for Genome Foundation Model Distillationcited by 0
  35. 2026Anchor-ECC: Local Integrity Checking for Watermarked LLM Outputs via Error-Correcting Codescited by 0