# Advancing Beyond Identification: Multi-bit Watermark for Large Language Models

arXiv: https://arxiv.org/abs/2308.00221 · HTML: https://mlsanity.com/paper/2308.00221/

## Key facts

- Cited by 57 papers in the map (data through 14 Oct 2026).
- Placed in Watermarking generated content and models, one of 12 directions in LLM safety and security.
- Builds on 28 of its 58 references that are in the map.
- First posted on arXiv 1 Aug 2023.

## Placed in

- LLM safety and security / [Watermarking generated content and models](https://mlsanity.com/safety/watermarking-generated-content-and-models/)

## Builds on (most cited first, 28 of 28)

- [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://mlsanity.com/paper/1910.10683/) (2019)
- [Llama 2: Open Foundation and Fine-Tuned Chat Models](https://mlsanity.com/paper/2307.09288/) (2023)
- [OPT: Open Pre-trained Transformer Language Models](https://mlsanity.com/paper/2205.01068/) (2022)
- [Pointer Sentinel Mixture Models](https://mlsanity.com/paper/1609.07843/) (2016)
- [DetectGPT: Zero-Shot Machine-Generated Text Detection using Probability Curvature](https://mlsanity.com/paper/2301.11305/) (2023)
- [A Watermark for Large Language Models](https://mlsanity.com/paper/2301.10226/) (2023)
- [Paraphrasing evades detectors of AI-generated text, but retrieval is an effective defense](https://mlsanity.com/paper/2303.13408/) (2023)
- [HiDDeN: Hiding Data With Deep Networks](https://mlsanity.com/paper/1807.09937/) (2018)
- [Robust Distortion-free Watermarks for Language Models](https://mlsanity.com/paper/2307.15593/) (2023)
- [The Stable Signature: Rooting Watermarks in Latent Diffusion Models](https://mlsanity.com/paper/2303.15435/) (2023)
- [Undetectable Watermarks for Language Models](https://mlsanity.com/paper/2306.09194/) (2023)
- [On the Reliability of Watermarks for Large Language Models](https://mlsanity.com/paper/2306.04634/) (2023)
- [A Recipe for Watermarking Diffusion Models](https://mlsanity.com/paper/2303.10137/) (2023)
- [Who Wrote this Code? Watermarking for Code Generation](https://mlsanity.com/paper/2305.15060/) (2023)
- [Adversarial Watermarking Transformer: Towards Tracing Text Provenance with Data Hiding](https://mlsanity.com/paper/2009.03015/) (2020)
- [M4: Multi-generator, Multi-domain, and Multi-lingual Black-Box Machine-Generated Text Detection](https://mlsanity.com/paper/2305.14902/) (2023)
- [Distortion Agnostic Deep Watermarking](https://mlsanity.com/paper/2001.04580/) (2020)
- [Protecting Intellectual Property of Language Generation APIs with Lexical Watermark](https://mlsanity.com/paper/2112.02701/) (2021)
- [Three Bricks to Consolidate Watermarks for Large Language Models](https://mlsanity.com/paper/2308.00113/) (2023)
- [Robust Multi-bit Natural Language Watermarking through Invariant Features](https://mlsanity.com/paper/2305.01904/) (2023)
- [Neural Linguistic Steganography](https://mlsanity.com/paper/1909.01496/) (2019)
- [Tracing Text Provenance via Context-Aware Lexical Substitution](https://mlsanity.com/paper/2112.07873/) (2021)
- [Generating Steganographic Text with LSTMs](https://mlsanity.com/paper/1705.10742/) (2017)
- [Perfectly Secure Steganography Using Minimum Entropy Coupling](https://mlsanity.com/paper/2210.14889/) (2022)
- [Analyzing the Digital Traces of Political Manipulation: The 2016 Russian Interference Twitter Campaign](https://mlsanity.com/paper/1802.04291/) (2018)
- [Paraphrastic Representations at Scale](https://mlsanity.com/paper/2104.15114/) (2021)
- [Propaganda and Misinformation on Facebook and Twitter during the Russian Invasion of Ukraine](https://mlsanity.com/paper/2212.00419/) (2022)
- [Binary Error-Correcting Codes with Minimal Noiseless Feedback](https://mlsanity.com/paper/2212.05673/) (2022)

## Built on by (most cited first, 30 of 57)

- [A Survey of Text Watermarking in the Era of Large Language Models](https://mlsanity.com/paper/2312.07913/) (2023)
- [On the Learnability of Watermarks for Language Models](https://mlsanity.com/paper/2312.04469/) (2023)
- [Watermarking Makes Language Models Radioactive](https://mlsanity.com/paper/2402.14904/) (2024)
- [Provably Robust Multi-bit Watermarking for AI-generated Text](https://mlsanity.com/paper/2401.16820/) (2024)
- [Towards Trustworthy Retrieval Augmented Generation for Large Language Models: A Survey](https://mlsanity.com/paper/2502.06872/) (2025)
- [Detecting AI-Generated Text: Factors Influencing Detectability with Current Methods](https://mlsanity.com/paper/2406.15583/) (2024)
- [Watermarking Techniques for Large Language Models: A Survey](https://mlsanity.com/paper/2409.00089/) (2024)
- [BiMark: Unbiased Multilayer Watermarking for Large Language Models](https://mlsanity.com/paper/2506.21602/) (2025)
- [StealthInk: A Multi-bit and Stealthy Watermark for Large Language Models](https://mlsanity.com/paper/2506.05502/) (2025)
- [Distributional Information Embedding: A Framework for Multi-bit Watermarking](https://mlsanity.com/paper/2501.16558/) (2025)
- [Enhancing LLM Watermark Resilience Against Both Scrubbing and Spoofing Attacks](https://mlsanity.com/paper/2507.06274/) (2025)
- [Watermarking Language Models through Language Models](https://mlsanity.com/paper/2411.05091/) (2024)
- [SAEMark: Steering Personalized Multilingual LLM Watermarks with Sparse Autoencoders](https://mlsanity.com/paper/2508.08211/) (2025)
- [SLICE: Semantic Latent Injection via Compartmentalized Embedding for Image Watermarking](https://mlsanity.com/paper/2603.12749/) (2026)
- [TextSeal: A Localized LLM Watermark for Provenance & Distillation Protection](https://mlsanity.com/paper/2605.12456/) (2026)
- [TRACE: A Two-Channel Robust Attribution Watermark via Complementary Embeddings for LLM-Agent Trajectories](https://mlsanity.com/paper/2607.08400/) (2026)
- [CredID: Credible Multi-Bit Watermark for Large Language Models Identification](https://mlsanity.com/paper/2412.03107/) (2024)
- [Majority Bit-Aware Watermarking For Large Language Models](https://mlsanity.com/paper/2508.03829/) (2025)
- [MC2Mark: Distortion-Free Multi-Bit Watermarking for Long Messages](https://mlsanity.com/paper/2602.14030/) (2026)
- [BiMarker: Enhancing Text Watermark Detection for Large Language Models with Bipolar Watermarks](https://mlsanity.com/paper/2501.12174/) (2025)
- [Watermarking Degrades Alignment in Language Models: Analysis and Mitigation](https://mlsanity.com/paper/2506.04462/) (2025)
- [Enhancing Watermarking Quality for LLMs via Contextual Generation States Awareness](https://mlsanity.com/paper/2506.07403/) (2025)
- [CATMark: A Context-Aware Thresholding Framework for Robust Cross-Task Watermarking in Large Language Models](https://mlsanity.com/paper/2510.02342/) (2025)
- [Distilling the Thought, Watermarking the Answer: A Principle Semantic Guided Watermark for Large Reasoning Models](https://mlsanity.com/paper/2601.05144/) (2026)
- [MirrorMark: Generalizable Mirrored Sampling for Multi-bit LLM Watermarking](https://mlsanity.com/paper/2601.22246/) (2026)
- [ArcMark: Distortion-Free Multi-Byte LLM Watermark via Optimal Transport](https://mlsanity.com/paper/2602.07235/) (2026)
- [SoK: Are Watermarks in LLMs Ready for Deployment?](https://mlsanity.com/paper/2506.05594/) (2025)
- [How Good is Post-Hoc Watermarking With Language Model Rephrasing?](https://mlsanity.com/paper/2512.16904/) (2025)
- [Beyond Raw Detection Scores: Markov-Informed Calibration for Boosting Machine-Generated Text Detection](https://mlsanity.com/paper/2602.08031/) (2026)
- [Undetectable Conversations Between AI Agents via Pseudorandom Noise-Resilient Key Exchange](https://mlsanity.com/paper/2604.04757/) (2026)

## Abstract (arXiv)

We show the viability of tackling misuses of large language models beyond the identification of machine-generated text. While existing zero-bit watermark methods focus on detection only, some malicious misuses demand tracing the adversary user for counteracting them. To address this, we propose Multi-bit Watermark via Position Allocation, embedding traceable multi-bit information during language model generation. Through allocating tokens onto different parts of the messages, we embed longer messages in high corruption settings without added latency. By independently embedding sub-units of messages, the proposed method outperforms the existing works in terms of robustness and latency. Leveraging the benefits of zero-bit watermarking, our method enables robust extraction of the watermark without any model access, embedding and extraction of long messages (≥ 32-bit) without finetuning, and maintaining text quality, while allowing zero-bit detection all at the same time. Code is released here: https://github.com/bangawayoo/mb-lm-watermarking