# Watermarking LLMs with Weight Quantization

arXiv: https://arxiv.org/abs/2310.11237 · HTML: https://mlsanity.com/paper/2310.11237/

## Key facts

- Cited by 22 papers in the map (data through 14 Oct 2026).
- Placed in Watermarking generated content and models, one of 12 directions in LLM safety and security.
- Builds on 24 of its 29 references that are in the map.
- First posted on arXiv 17 Oct 2023.

## Placed in

- LLM safety and security / [Watermarking generated content and models](https://mlsanity.com/safety/watermarking-generated-content-and-models/)

## Builds on (most cited first, 24 of 24)

- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [GPT-4 Technical Report](https://mlsanity.com/paper/2303.08774/) (2023)
- [LLaMA: Open and Efficient Foundation Language Models](https://mlsanity.com/paper/2302.13971/) (2023)
- [SQuAD: 100,000+ Questions for Machine Comprehension of Text](https://mlsanity.com/paper/1606.05250/) (2016)
- [PubMedQA: A Dataset for Biomedical Research Question Answering](https://mlsanity.com/paper/1909.06146/) (2019)
- [Hierarchical Neural Story Generation](https://mlsanity.com/paper/1805.04833/) (2018)
- [LLM.int8(): 8-bit Matrix Multiplication for Transformers at Scale](https://mlsanity.com/paper/2208.07339/) (2022)
- [DetectGPT: Zero-Shot Machine-Generated Text Detection using Probability Curvature](https://mlsanity.com/paper/2301.11305/) (2023)
- [A Watermark for Large Language Models](https://mlsanity.com/paper/2301.10226/) (2023)
- [Defending Against Neural Fake News](https://mlsanity.com/paper/1905.12616/) (2019)
- [Q-BERT: Hessian Based Ultra Low Precision Quantization of BERT](https://mlsanity.com/paper/1909.05840/) (2019)
- [Weight Poisoning Attacks on Pre-trained Models](https://mlsanity.com/paper/2004.06660/) (2020)
- [Q8BERT: Quantized 8Bit BERT](https://mlsanity.com/paper/1910.06188/) (2019)
- [Backdoor Attacks on Pre-trained Models by Layerwise Weight Poisoning](https://mlsanity.com/paper/2108.13888/) (2021)
- [Real or Fake? Learning to Discriminate Machine from Human Generated Text](https://mlsanity.com/paper/1906.03351/) (2019)
- [Protecting Intellectual Property of Language Generation APIs with Lexical Watermark](https://mlsanity.com/paper/2112.02701/) (2021)
- [Neural Linguistic Steganography](https://mlsanity.com/paper/1909.01496/) (2019)
- [CATER: Intellectual Property Protection on Text Generation APIs via Conditional Watermarks](https://mlsanity.com/paper/2209.08773/) (2022)
- [A Statistical Framework for Low-bitwidth Training of Deep Neural Networks](https://mlsanity.com/paper/2010.14298/) (2020)
- [Red Alarm for Pre-trained Models: Universal Vulnerability to Neuron-Level Backdoor Attacks](https://mlsanity.com/paper/2101.06969/) (2021)
- [Generating Steganographic Text with LSTMs](https://mlsanity.com/paper/1705.10742/) (2017)
- [Origin Tracing and Detecting of LLMs](https://mlsanity.com/paper/2304.14072/) (2023)
- [Towards Fully 8-bit Integer Inference for the Transformer Model](https://mlsanity.com/paper/2009.08034/) (2020)
- [Towards Near-imperceptible Steganographic Text](https://mlsanity.com/paper/1907.06679/) (2019)

## Built on by (most cited first, 22 of 22)

- [Model Merging in LLMs, MLLMs, and Beyond: Methods, Theories, Applications and Opportunities](https://mlsanity.com/paper/2408.07666/) (2024)
- [A Survey of Recent Backdoor Attacks and Defenses in Large Language Models](https://mlsanity.com/paper/2406.06852/) (2024)
- [Optimizing watermarks for large language models](https://mlsanity.com/paper/2312.17295/) (2023)
- [Watermarking Techniques for Large Language Models: A Survey](https://mlsanity.com/paper/2409.00089/) (2024)
- [Have You Merged My Model? On The Robustness of Large Language Model IP Protection Methods Against Model Merging](https://mlsanity.com/paper/2404.05188/) (2024)
- [Mark Your LLM: Detecting the Misuse of Open-Source Large Language Models via Watermarking](https://mlsanity.com/paper/2503.04636/) (2025)
- [EmMark: Robust Watermarks for IP Protection of Embedded Quantized Large Language Models](https://mlsanity.com/paper/2402.17938/) (2024)
- [Turning Your Strength into Watermark: Watermarking Large Language Model via Knowledge Injection](https://mlsanity.com/paper/2311.09535/) (2023)
- [Watermarking Language Models through Language Models](https://mlsanity.com/paper/2411.05091/) (2024)
- [Model Unmerging: Making Your Models Unmergeable for Secure Model Sharing](https://mlsanity.com/paper/2509.01548/) (2025)
- [BitMark: Watermarking Bitwise Autoregressive Image Generative Models](https://mlsanity.com/paper/2506.21209/) (2025)
- [EditMark: Watermarking Large Language Models based on Model Editing](https://mlsanity.com/paper/2510.16367/) (2025)
- [Watermarking Large Language Models and the Generated Content: Opportunities and Challenges](https://mlsanity.com/paper/2410.19096/) (2024)
- [SELF: A Robust Singular Value and Eigenvalue Approach for LLM Fingerprinting](https://mlsanity.com/paper/2512.03620/) (2025)
- [Signal Watermark on Large Language Models](https://mlsanity.com/paper/2410.06545/) (2024)
- [Provable Model Provenance Set for Large Language Models](https://mlsanity.com/paper/2602.00772/) (2026)
- [Protecting Copyright of Medical Pre-trained Language Models: Training-Free Backdoor Model Watermarking](https://mlsanity.com/paper/2409.10570/) (2024)
- [WaterDrum: Watermarking for Data-centric Unlearning Metric](https://mlsanity.com/paper/2505.05064/) (2025)
- [Watermarking Large Language Models in Europe: Interpreting the AI Act in Light of Technology](https://mlsanity.com/paper/2511.03641/) (2025)
- [Functional Subspace Watermarking for Large Language Models](https://mlsanity.com/paper/2603.18793/) (2026)
- [Stemma: Induced Decision Regions Reveal LLM Provenance](https://mlsanity.com/paper/2607.25880/) (2026)
- [WoE Wrote It? Watermarking Mixture-of-Experts LLMs for Black-Box Text Provenance](https://mlsanity.com/paper/2608.29151/) (2026)

## Abstract (arXiv)

Abuse of large language models reveals high risks as large language models are being deployed at an astonishing speed. It is important to protect the model weights to avoid malicious usage that violates licenses of open-source large language models. This paper proposes a novel watermarking strategy that plants watermarks in the quantization process of large language models without pre-defined triggers during inference. The watermark works when the model is used in the fp32 mode and remains hidden when the model is quantized to int8, in this way, the users can only inference the model without further supervised fine-tuning of the model. We successfully plant the watermark into open-source large language model weights including GPT-Neo and LLaMA. We hope our proposed method can provide a potential direction for protecting model weights in the era of large language model applications.