# Robust Multi-bit Natural Language Watermarking through Invariant Features

arXiv: https://arxiv.org/abs/2305.01904 · HTML: https://mlsanity.com/paper/2305.01904/

## Key facts

- Cited by 82 papers in the map (data through 14 Oct 2026).
- Placed in Watermarking generated content and models, one of 12 directions in LLM safety and security.
- Builds on 20 of its 55 references that are in the map.
- First posted on arXiv 3 May 2023.

## Placed in

- LLM safety and security / [Watermarking generated content and models](https://mlsanity.com/safety/watermarking-generated-content-and-models/)

## Builds on (most cited first, 20 of 20)

- [Pointer Sentinel Mixture Models](https://mlsanity.com/paper/1609.07843/) (2016)
- [A Watermark for Large Language Models](https://mlsanity.com/paper/2301.10226/) (2023)
- [Is BERT Really Robust? A Strong Baseline for Natural Language Attack on Text Classification and Entailment](https://mlsanity.com/paper/1907.11932/) (2019)
- [HiDDeN: Hiding Data With Deep Networks](https://mlsanity.com/paper/1807.09937/) (2018)
- [BAE: BERT-based Adversarial Examples for Text Classification](https://mlsanity.com/paper/2004.01970/) (2020)
- [Generative Language Models and Automated Influence Operations: Emerging Threats and Potential Mitigations](https://mlsanity.com/paper/2301.04246/) (2023)
- [Pathologies of Neural Models Make Interpretations Difficult](https://mlsanity.com/paper/1804.07781/) (2018)
- [Contextualized Perturbation for Textual Adversarial Attack](https://mlsanity.com/paper/2009.07502/) (2020)
- [Adversarial Watermarking Transformer: Towards Tracing Text Provenance with Data Hiding](https://mlsanity.com/paper/2009.03015/) (2020)
- [Distortion Agnostic Deep Watermarking](https://mlsanity.com/paper/2001.04580/) (2020)
- [Reevaluating Adversarial Examples in Natural Language](https://mlsanity.com/paper/2004.14174/) (2020)
- [Protecting Intellectual Property of Language Generation APIs with Lexical Watermark](https://mlsanity.com/paper/2112.02701/) (2021)
- [Neural Linguistic Steganography](https://mlsanity.com/paper/1909.01496/) (2019)
- [CATER: Intellectual Property Protection on Text Generation APIs via Conditional Watermarks](https://mlsanity.com/paper/2209.08773/) (2022)
- [TextAttack: A Framework for Adversarial Attacks, Data Augmentation, and Adversarial Training in NLP](https://mlsanity.com/paper/2005.05909/) (2020)
- [Tracing Text Provenance via Context-Aware Lexical Substitution](https://mlsanity.com/paper/2112.07873/) (2021)
- [Generating Steganographic Text with LSTMs](https://mlsanity.com/paper/1705.10742/) (2017)
- [Frustratingly Easy Edit-based Linguistic Steganography with a Masked Language Model](https://mlsanity.com/paper/2104.09833/) (2021)
- [A Robust Blind Watermarking Using Convolutional Neural Network](https://mlsanity.com/paper/1704.03248/) (2017)
- [A Robust Image Watermarking System Based on Deep Neural Networks](https://mlsanity.com/paper/1908.11331/) (2019)

## Built on by (most cited first, 30 of 82)

- [Challenges and Applications of Large Language Models](https://mlsanity.com/paper/2307.10169/) (2023)
- [Can LLM-Generated Misinformation Be Detected?](https://mlsanity.com/paper/2309.13788/) (2023)
- [Monitoring AI-Modified Content at Scale: A Case Study on the Impact of ChatGPT on AI Conference Peer Reviews](https://mlsanity.com/paper/2403.07183/) (2024)
- [Mapping the Increasing Use of LLMs in Scientific Papers](https://mlsanity.com/paper/2404.01268/) (2024)
- [A Semantic Invariant Robust Watermark for Large Language Models](https://mlsanity.com/paper/2310.06356/) (2023)
- [A Survey of Text Watermarking in the Era of Large Language Models](https://mlsanity.com/paper/2312.07913/) (2023)
- [A Robust Semantics-based Watermark for Large Language Model against Paraphrasing](https://mlsanity.com/paper/2311.08721/) (2023)
- [REMARK-LLM: A Robust and Efficient Watermarking Framework for Generative Large Language Models](https://mlsanity.com/paper/2310.12362/) (2023)
- [Detecting Multimedia Generated by Large AI Models: A Survey](https://mlsanity.com/paper/2402.00045/) (2024)
- [An Entropy-based Text Watermarking Detection Method](https://mlsanity.com/paper/2403.13485/) (2024)
- [A Survey on Detection of LLMs-Generated Content](https://mlsanity.com/paper/2310.15654/) (2023)
- [Adaptive Text Watermark for Large Language Models](https://mlsanity.com/paper/2401.13927/) (2024)
- [An Unforgeable Publicly Verifiable Watermark for Large Language Models](https://mlsanity.com/paper/2307.16230/) (2023)
- [Preventing Language Models From Hiding Their Reasoning](https://mlsanity.com/paper/2310.18512/) (2023)
- [Advancing Beyond Identification: Multi-bit Watermark for Large Language Models](https://mlsanity.com/paper/2308.00221/) (2023)
- [Towards Codable Watermarking for Injecting Multi-bits Information to LLMs](https://mlsanity.com/paper/2307.15992/) (2023)
- [k-SemStamp: A Clustering-Based Semantic Watermark for Detection of Machine-Generated Text](https://mlsanity.com/paper/2402.11399/) (2024)
- [Provably Robust Multi-bit Watermarking for AI-generated Text](https://mlsanity.com/paper/2401.16820/) (2024)
- [GumbelSoft: Diversified Language Model Watermarking via the GumbelMax-trick](https://mlsanity.com/paper/2402.12948/) (2024)
- [Token-Specific Watermarking with Enhanced Detectability and Semantic Coherence for Large Language Models](https://mlsanity.com/paper/2402.18059/) (2024)
- [Towards Trustworthy Retrieval Augmented Generation for Large Language Models: A Survey](https://mlsanity.com/paper/2502.06872/) (2025)
- [Optimizing watermarks for large language models](https://mlsanity.com/paper/2312.17295/) (2023)
- [Deconstructing The Ethics of Large Language Models from Long-standing Issues to New-emerging Dilemmas: A Survey](https://mlsanity.com/paper/2406.05392/) (2024)
- [Detecting AI-Generated Text: Factors Influencing Detectability with Current Methods](https://mlsanity.com/paper/2406.15583/) (2024)
- [Permute-and-Flip: An optimally stable and watermarkable decoder for LLMs](https://mlsanity.com/paper/2402.05864/) (2024)
- [Theoretically Grounded Framework for LLM Watermarking: A Distribution-Adaptive Approach](https://mlsanity.com/paper/2410.02890/) (2024)
- [Double-I Watermark: Protecting Model Copyright for LLM Fine-tuning](https://mlsanity.com/paper/2402.14883/) (2024)
- [BiMark: Unbiased Multilayer Watermarking for Large Language Models](https://mlsanity.com/paper/2506.21602/) (2025)
- [Ten Words Only Still Help: Improving Black-Box AI-Generated Text Detection via Proxy-Guided Efficient Re-Sampling](https://mlsanity.com/paper/2402.09199/) (2024)
- [Copyright Protection for Large Language Models: A Survey of Methods, Challenges, and Trends](https://mlsanity.com/paper/2508.11548/) (2025)

## Abstract (arXiv)

Recent years have witnessed a proliferation of valuable original natural language contents found in subscription-based media outlets, web novel platforms, and outputs of large language models. However, these contents are susceptible to illegal piracy and potential misuse without proper security measures. This calls for a secure watermarking system to guarantee copyright protection through leakage tracing or ownership identification. To effectively combat piracy and protect copyrights, a multi-bit watermarking framework should be able to embed adequate bits of information and extract the watermarks in a robust manner despite possible corruption. In this work, we explore ways to advance both payload and robustness by following a well-known proposition from image watermarking and identify features in natural language that are invariant to minor corruption. Through a systematic analysis of the possible sources of errors, we further propose a corruption-resistant infill model. Our full method improves upon the previous work on robustness by +16.8% point on average on four datasets, three corruption types, and two corruption ratios. Code available at https://github.com/bangawayoo/nlp-watermarking.