mlsanityData to 7 Oct 2026

A Survey of Text Watermarking in the Era of Large Language Models

arXiv 2312.07913 ↗First posted 13 Dec 2023cs.CLcited by 102

On the map: LLM safety and security → Watermarking generated content and models

Abstract (arXiv)

Text watermarking algorithms are crucial for protecting the copyright of textual content. Historically, their capabilities and application scenarios were limited. However, recent advancements in large language models (LLMs) have revolutionized these techniques. LLMs not only enhance text watermarking algorithms with their advanced abilities but also create a need for employing these algorithms to protect their own copyrights or prevent potential misuse. This paper conducts a comprehensive survey of the current state of text watermarking technology, covering four main aspects: (1) an overview and comparison of different text watermarking techniques; (2) evaluation methods for text watermarking algorithms, including their detectability, impact on text or LLM quality, robustness under target or untargeted attacks; (3) potential application scenarios for text watermarking technology; (4) current challenges and future directions for text watermarking. This survey aims to provide researchers with a thorough understanding of text watermarking technology in the era of LLM, thereby promoting its further advancement.

Key facts. Cited by 102 papers in the map (data through 7 Oct 2026). Placed in Watermarking generated content and models, one of 12 directions in LLM safety and security. Builds on 72 of its 141 references that are in the map. First posted on arXiv 13 Dec 2023.

Builds on

72 of 141 references.

Closest ancestors · 24

Same direction, newest first

  1. 2023On the Learnability of Watermarks for Language Models7 Dec
  2. 2023Mark My Words: Analyzing and Evaluating Language Model Watermarks1 Dec
  3. 2023A Robust Semantics-based Watermark for Large Language Model against Paraphrasing15 Nov
  4. 2023Publicly-Detectable Watermarking for Language Models27 Oct
  5. 2023REMARK-LLM: A Robust and Efficient Watermarking Framework for Generative Large Language Models18 Oct
  6. 2023CodeMark: Imperceptible Watermarking for Code Datasets against Neural Code Completion Models28 Aug
  7. 2023Advancing Beyond Identification: Multi-bit Watermark for Large Language Models1 Aug
  8. 2023An Unforgeable Publicly Verifiable Watermark for Large Language Models30 Jul
  9. 2023Towards Codable Watermarking for Injecting Multi-bits Information to LLMs29 Jul
  10. 2023Robust Distortion-free Watermarks for Language Models28 Jul
  11. 2023Watermarking Conditional Text Generation for AI Detection: Unveiling Challenges and a Semantic-Aware Watermark Remedy25 Jul
  12. 2023Provable Robust Watermarking for AI-Generated Text30 Jun
12 more
  1. 2023Who Wrote this Code? Watermarking for Code Generation24 May
  2. 2023Watermarking Text Data on Large Language Models for Dataset Copyright22 May
  3. 2023Are You Copying My Model? Protecting the Copyright of Large Language Models for EaaS via Backdoor Watermark17 May
  4. 2023Watermarking Text Generated by Black-Box Language Models14 May
  5. 2023Robust Multi-bit Natural Language Watermarking through Invariant Features3 May
  6. 2023Paraphrasing evades detectors of AI-generated text, but retrieval is an effective defense23 Mar
  7. 2023Can AI-Generated Text be Reliably Detected?17 Mar
  8. 2023Protecting Language Generation Models via Invisible Watermarking6 Feb
  9. 2022Distillation-Resistant Watermarking for Model Protection in NLP7 Oct
  10. 2022CATER: Intellectual Property Protection on Text Generation APIs via Conditional Watermarks19 Sep
  11. 2021Tracing Text Provenance via Context-Aware Lexical Substitution15 Dec
  12. 2021Protecting Intellectual Property of Language Generation APIs with Lexical Watermark5 Dec

Other references · 48

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
  3. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
  4. 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,049
  5. 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,393
  6. 2021Evaluating Large Language Models Trained on Codecited by 8,053
  7. 2019BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehensioncited by 5,874
  8. 2019BERTScore: Evaluating Text Generation with BERTcited by 5,462
  9. 2018HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answeringcited by 3,686
  10. 2022OPT: Open Pre-trained Transformer Language Modelscited by 3,464
  11. 2021TruthfulQA: Measuring How Models Mimic Human Falsehoodscited by 2,919
  12. 2017TriviaQA: A Large Scale Distantly Supervised Challenge Dataset for Reading Comprehensioncited by 2,895
36 more
  1. 2019The Curious Case of Neural Text Degenerationcited by 2,881
  2. 2016Categorical Reparameterization with Gumbel-Softmaxcited by 2,708
  3. 2023Sparks of Artificial General Intelligence: Early experiments with GPT-4cited by 2,579
  4. 2022No Language Left Behind: Scaling Human-Centered Machine Translationcited by 1,290
  5. 2022LaMDA: Language Models for Dialog Applicationscited by 1,218
  6. 2015Teaching Machines to Read and Comprehendcited by 1,039
  7. 2020Multilingual Denoising Pre-training for Neural Machine Translationcited by 1,028
  8. 2022Red Teaming Language Models with Language Modelscited by 983
  9. 2022CodeGen: An Open Large Language Model for Code with Multi-Turn Program Synthesiscited by 876
  10. 2023Large Language Modelscited by 774
  11. 2023AlpacaFarm: A Simulation Framework for Methods that Learn from Human Feedbackcited by 738
  12. 2023DetectGPT: Zero-Shot Machine-Generated Text Detection using Probability Curvaturecited by 705
  13. 2019ELI5: Long Form Question Answeringcited by 567
  14. 2018Universal Sentence Encodercited by 485
  15. 2023Benchmarking Large Language Models for News Summarizationcited by 444
  16. 2016A Corpus and Evaluation Framework for Deeper Understanding of Commonsense Storiescited by 413
  17. 2023A Survey on Large Language Model (LLM) Security and Privacy: The Good, the Bad, and the Uglycited by 380
  18. 2022A Systematic Evaluation of Large Language Models of Codecited by 360
  19. 2023How Good Are GPT Models at Machine Translation? A Comprehensive Evaluationcited by 344
  20. 2023LEVER: Learning to Verify Language-to-Code Generation with Executioncited by 228
  21. 2017Embedding Watermarks into Deep Neural Networkscited by 227
  22. 2019SPoC: Search-based Pseudocode to Codecited by 176
  23. 2023Can LLM-Generated Misinformation Be Detected?cited by 173
  24. 2023Can Sensitive Information Be Deleted From LLMs? Objectives for Defending Against Extraction Attackscited by 156
  25. 2020Adversarial Watermarking Transformer: Towards Tracing Text Provenance with Data Hidingcited by 121
  26. 2022Language Models that Seek for Knowledge: Modular Search & Generation for Dialogue and Prompt Completioncited by 102
  27. 2023Combating Misinformation in the Age of LLMs: Opportunities and Challengescited by 94
  28. 2023Are LLMs All You Need for Task-Oriented Dialogue?cited by 52
  29. 2021CoProtector: Protect Open-Source Code against Unauthorized Training Usage with Data Poisoningcited by 39
  30. 2023Did You Train on My Dataset? Towards Public Dataset Protection with Clean-Label Backdoor Watermarkingcited by 37
  31. 2020Unsupervised Text Generation by Learning from Searchcited by 36
  32. 2023HowkGPT: Investigating the Detection of ChatGPT-generated University Student Homework through Context-Aware Perplexity Analysiscited by 30
  33. 2023Model Leeching: An Extraction Attack Targeting LLMscited by 22
  34. 2023WatME: Towards Lossless Watermarking Through Lexical Redundancycited by 22
  35. 2022Text Revision by On-the-Fly Representation Optimizationcited by 17
  36. 2023A Sequence-to-Sequence&Set Model for Text-to-Table Generationcited by 16

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 24; 2025: 51; 2026: 27.

102 papers.

Sort:

By area:

  1. 2024MarkLLM: An Open-Source Toolkit for LLM Watermarkingcited by 80
  2. 2024Watermark Stealing in Large Language Modelscited by 79
  3. 2024Visibility into AI Agentscited by 77
  4. 2024An Entropy-based Text Watermarking Detection Methodcited by 61
  5. 2024SoK: Watermarking for AI-Generated Contentcited by 43
  6. 2024TIS-DPO: Token-level Importance Sampling for Direct Preference Optimization With Estimated Weightscited by 42
  7. 2024GumbelSoft: Diversified Language Model Watermarking via the GumbelMax-trickcited by 31
  8. 2024On the Challenges and Opportunities in Generative AIcited by 31
  9. 2025Towards Trustworthy Retrieval Augmented Generation for Large Language Models: A Surveycited by 29
  10. 2024Can Watermarks Survive Translation? On the Cross-lingual Consistency of Text Watermark for Large Language Modelscited by 27
  11. 2024Detecting AI-Generated Text: Factors Influencing Detectability with Current Methodscited by 23
  12. 2025Frontier AI's Impact on the Cybersecurity Landscapecited by 21
38 more
  1. 2024IDs for AI Systemscited by 18
  2. 2025BiMark: Unbiased Multilayer Watermarking for Large Language Modelscited by 18
  3. 2025A Survey on Model Extraction Attacks and Defenses for Large Language Modelscited by 18
  4. 2024Theoretically Grounded Framework for LLM Watermarking: A Distribution-Adaptive Approachcited by 17
  5. 2025VideoMark: A Distortion-Free Robust Watermarking Framework for Video Diffusion Modelscited by 16
  6. 2025Mark Your LLM: Detecting the Misuse of Open-Source Large Language Models via Watermarkingcited by 14
  7. 2025SoK: Large Language Model Copyright Auditing via Fingerprintingcited by 14
  8. 2025Towards Watermarking of Open-Source LLMscited by 11
  9. 2024Can Watermarked LLMs be Identified by Users via Crafted Prompts?cited by 10
  10. 2024WaterSeeker: Pioneering Efficient Detection of Watermarked Segments in Large Documentscited by 9
  11. 2025Optimized Couplings for Watermarking Large Language Modelscited by 9
  12. 2025In-Context Watermarks for Large Language Modelscited by 9
  13. 2024Robust Multi-bit Text Watermark with LLM-based Paraphraserscited by 8
  14. 2025DATABench: Evaluating Dataset Auditing in Deep Learning from an Adversarial Perspectivecited by 8
  15. 2025Enhancing LLM Watermark Resilience Against Both Scrubbing and Spoofing Attackscited by 8
  16. 2024Watermarking Language Models through Language Modelscited by 7
  17. 2024Watermark under Fire: A Robustness Evaluation of LLM Watermarkingcited by 7
  18. 2025First-Place Solution to NeurIPS 2024 Invisible Watermark Removal Challengecited by 7
  19. 2025Position: LLM Watermarking Should Align Stakeholders' Incentives for Practical Adoptioncited by 7
  20. 2024WaterPool: A Watermark Mitigating Trade-offs among Imperceptibility, Efficacy and Robustnesscited by 6
  21. 2025Diversity Boosts AI-Generated Text Detectioncited by 6
  22. 2024FoldMark: Protecting Protein Generative Models with Watermarkingcited by 5
  23. 2025Limits of LLM Text Detectors in Educationcited by 5
  24. 2025MarkDiffusion: An Open-Source Toolkit for Generative Watermarking of Latent Diffusion Modelscited by 5
  25. 2024On Evaluating The Performance of Watermarked Machine-Generated Texts Under Adversarial Attackscited by 4
  26. 2025Robustness Assessment and Enhancement of Text Watermarking for Google's SynthIDcited by 4
  27. 2025Paladin: Defending LLM-enabled Phishing Emails with a New Trigger-Tag Paradigmcited by 4
  28. 2025DMark: Order-Agnostic Watermarking for Diffusion Large Language Modelscited by 4
  29. 2026PASA: A Principled Embedding-Space Watermarking Approach for LLM-Generated Text under Semantic-Invariant Attackscited by 4
  30. 2024Evaluating Durability: Benchmark Insights into Multimodal Watermarkingcited by 3
  31. 2024CredID: Credible Multi-Bit Watermark for Large Language Models Identificationcited by 3
  32. 2025BiMarker: Enhancing Text Watermark Detection for Large Language Models with Bipolar Watermarkscited by 3
  33. 2025Video Signature: Implicit Watermarking for Video Diffusion Modelscited by 3
  34. 2025Sword and Shield: Uses and Strategies of LLMs in Navigating Disinformationcited by 3
  35. 2025Multi-use LLM Watermarking and the False Detection Problemcited by 3
  36. 2025Majority Bit-Aware Watermarking For Large Language Modelscited by 3
  37. 2025Data Provenance Auditing of Fine-Tuned Large Language Models with a Text-Preserving Techniquecited by 3
  38. 2025Every Language Model Has a Forgery-Resistant Signaturecited by 3

50 most cited of 102 All 102 (JSON)