mlsanityData to 14 Oct 2026

Robust Multi-bit Natural Language Watermarking through Invariant Features

arXiv 2305.01904 ↗First posted 3 May 2023cs.CL cs.AIcited by 82

On the map: LLM safety and security → Watermarking generated content and models

Abstract (arXiv)

Recent years have witnessed a proliferation of valuable original natural language contents found in subscription-based media outlets, web novel platforms, and outputs of large language models. However, these contents are susceptible to illegal piracy and potential misuse without proper security measures. This calls for a secure watermarking system to guarantee copyright protection through leakage tracing or ownership identification. To effectively combat piracy and protect copyrights, a multi-bit watermarking framework should be able to embed adequate bits of information and extract the watermarks in a robust manner despite possible corruption. In this work, we explore ways to advance both payload and robustness by following a well-known proposition from image watermarking and identify features in natural language that are invariant to minor corruption. Through a systematic analysis of the possible sources of errors, we further propose a corruption-resistant infill model. Our full method improves upon the previous work on robustness by +16.8% point on average on four datasets, three corruption types, and two corruption ratios. Code available at https://github.com/bangawayoo/nlp-watermarking.

Key facts. Cited by 82 papers in the map (data through 14 Oct 2026). Placed in Watermarking generated content and models, one of 12 directions in LLM safety and security. Builds on 20 of its 55 references that are in the map. First posted on arXiv 3 May 2023.

Builds on

20 of 55 references.

Closest ancestors · 4

Same direction, newest first

  1. 2023A Watermark for Large Language Models24 Jan
  2. 2022CATER: Intellectual Property Protection on Text Generation APIs via Conditional Watermarks19 Sep
  3. 2021Tracing Text Provenance via Context-Aware Lexical Substitution15 Dec
  4. 2021Protecting Intellectual Property of Language Generation APIs with Lexical Watermark5 Dec

Other references · 16

  1. 2016Pointer Sentinel Mixture Modelscited by 3,096
  2. 2019Is BERT Really Robust? A Strong Baseline for Natural Language Attack on Text Classification and Entailmentcited by 671
  3. 2018HiDDeN: Hiding Data With Deep Networkscited by 397
  4. 2020BAE: BERT-based Adversarial Examples for Text Classificationcited by 316
  5. 2023Generative Language Models and Automated Influence Operations: Emerging Threats and Potential Mitigationscited by 158
  6. 2018Pathologies of Neural Models Make Interpretations Difficultcited by 140
  7. 2020Contextualized Perturbation for Textual Adversarial Attackcited by 136
  8. 2020Adversarial Watermarking Transformer: Towards Tracing Text Provenance with Data Hidingcited by 130
  9. 2020Distortion Agnostic Deep Watermarkingcited by 90
  10. 2020Reevaluating Adversarial Examples in Natural Languagecited by 85
  11. 2019Neural Linguistic Steganographycited by 78
  12. 2020TextAttack: A Framework for Adversarial Attacks, Data Augmentation, and Adversarial Training in NLPcited by 45
4 more
  1. 2017Generating Steganographic Text with LSTMscited by 44
  2. 2021Frustratingly Easy Edit-based Linguistic Steganography with a Masked Language Modelcited by 30
  3. 2017A Robust Blind Watermarking Using Convolutional Neural Networkcited by 11
  4. 2019A Robust Image Watermarking System Based on Deep Neural Networkscited by 5

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 14; 2024: 33; 2025: 29; 2026: 6.

82 papers.

Sort:

By area:

  1. 2023Challenges and Applications of Large Language Modelscited by 254
  2. 2023Can LLM-Generated Misinformation Be Detected?cited by 177
  3. 2024Monitoring AI-Modified Content at Scale: A Case Study on the Impact of ChatGPT on AI Conference Peer Reviewscited by 164
  4. 2024Mapping the Increasing Use of LLMs in Scientific Paperscited by 118
  5. 2023A Semantic Invariant Robust Watermark for Large Language Modelscited by 112
  6. 2023A Survey of Text Watermarking in the Era of Large Language Modelscited by 112
  7. 2023A Robust Semantics-based Watermark for Large Language Model against Paraphrasingcited by 82
  8. 2023REMARK-LLM: A Robust and Efficient Watermarking Framework for Generative Large Language Modelscited by 78
  9. 2024Detecting Multimedia Generated by Large AI Models: A Surveycited by 76
  10. 2024An Entropy-based Text Watermarking Detection Methodcited by 75
  11. 2023A Survey on Detection of LLMs-Generated Contentcited by 71
  12. 2024Adaptive Text Watermark for Large Language Modelscited by 67
38 more
  1. 2023An Unforgeable Publicly Verifiable Watermark for Large Language Modelscited by 65
  2. 2023Preventing Language Models From Hiding Their Reasoningcited by 65
  3. 2023Advancing Beyond Identification: Multi-bit Watermark for Large Language Modelscited by 57
  4. 2023Towards Codable Watermarking for Injecting Multi-bits Information to LLMscited by 54
  5. 2024k-SemStamp: A Clustering-Based Semantic Watermark for Detection of Machine-Generated Textcited by 51
  6. 2024Provably Robust Multi-bit Watermarking for AI-generated Textcited by 47
  7. 2024GumbelSoft: Diversified Language Model Watermarking via the GumbelMax-trickcited by 37
  8. 2024Token-Specific Watermarking with Enhanced Detectability and Semantic Coherence for Large Language Modelscited by 34
  9. 2025Towards Trustworthy Retrieval Augmented Generation for Large Language Models: A Surveycited by 30
  10. 2023Optimizing watermarks for large language modelscited by 25
  11. 2024Deconstructing The Ethics of Large Language Models from Long-standing Issues to New-emerging Dilemmas: A Surveycited by 23
  12. 2024Detecting AI-Generated Text: Factors Influencing Detectability with Current Methodscited by 23
  13. 2024Permute-and-Flip: An optimally stable and watermarkable decoder for LLMscited by 22
  14. 2024Theoretically Grounded Framework for LLM Watermarking: A Distribution-Adaptive Approachcited by 22
  15. 2024Double-I Watermark: Protecting Model Copyright for LLM Fine-tuningcited by 21
  16. 2025BiMark: Unbiased Multilayer Watermarking for Large Language Modelscited by 21
  17. 2024Ten Words Only Still Help: Improving Black-Box AI-Generated Text Detection via Proxy-Guided Efficient Re-Samplingcited by 20
  18. 2025Copyright Protection for Large Language Models: A Survey of Methods, Challenges, and Trendscited by 20
  19. 2025Improving Your Model Ranking on Chatbot Arena by Vote Riggingcited by 17
  20. 2024A Watermark for Black-Box Language Modelscited by 16
  21. 2024Waterfall: Framework for Robust and Scalable Text Watermarking and Provenance for LLMscited by 15
  22. 2024SoK: On the Role and Future of AIGC Watermarking in the Era of Gen-AIcited by 14
  23. 2025Survey on AI-Generated Media Detection: From Non-MLLM to MLLMcited by 12
  24. 2023New Evaluation Metrics Capture Quality Degradation due to LLM Watermarkingcited by 11
  25. 2024Multi-Bit Distortion-Free Watermarking for Large Language Modelscited by 11
  26. 2024Uncovering LLM-Generated Code: A Zero-Shot Synthetic Code Detector via Code Rewritingcited by 11
  27. 2024Is Contrasting All You Need? Contrastive Learning for the Detection and Attribution of AI-generated Textcited by 11
  28. 2024Robust Multi-bit Text Watermark with LLM-based Paraphraserscited by 10
  29. 2025Defending LLM Watermarking Against Spoofing Attacks with Contrastive Representation Learningcited by 10
  30. 2025In-Context Watermarks for Large Language Modelscited by 10
  31. 2024WaterJudge: Quality-Detection Trade-off when Watermarking Large Language Modelscited by 9
  32. 2024An Auditing Test To Detect Behavioral Shift in Language Modelscited by 9
  33. 2025Distributional Information Embedding: A Framework for Multi-bit Watermarkingcited by 9
  34. 2025OpenTuringBench: An Open-Model-based Benchmark and Framework for Machine-Generated Text Detection and Attributioncited by 9
  35. 2024'Quis custodiet ipsos custodes?' Who will watch the watchmen? On Detecting AI-generated peer-reviewscited by 8
  36. 2024Inevitable Trade-off between Watermark Strength and Speculative Sampling Efficiency for Language Modelscited by 8
  37. 2025Authorship Attribution in Multilingual Machine-Generated Textscited by 8
  38. 2025Human Texts Are Outliers: Detecting LLM-generated Texts via Out-of-distribution Detectioncited by 8

50 most cited of 82 All 82 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2025DERMARK: A Dynamic, Efficient and Robust Multi-bit Watermark for Large Language Models
  2. 2024Let Watermarks Speak: A Robust and Unforgeable Watermark for Language Models
  3. 2024Watermarking Language Models with Error Correcting Codes
  4. 2026Unforgeable Watermarks for Language Models via Robust Signatures
  5. 2026SimpleMark: Fast Multi-Bit Text Watermarking under f -Divergence Constraints
  6. 2026Block-wise Codeword Embedding for Reliable Multi-bit Text Watermarking
  7. 2023Robust Distortion-free Watermarks for Language Models
  8. 2024Watermarking Language Models for Many Adaptive Users