# A Statistical Framework of Watermarks for Large Language Models: Pivot, Detection Efficiency and Optimal Rules

arXiv: https://arxiv.org/abs/2404.01245 · HTML: https://mlsanity.com/paper/2404.01245/

## Key facts

- Cited by 57 papers in the map (data through 14 Oct 2026).
- Placed in Watermarking generated content and models, one of 12 directions in LLM safety and security.
- Builds on 3 of its 13 references that are in the map.
- First posted on arXiv 1 Apr 2024.

## Placed in

- LLM safety and security / [Watermarking generated content and models](https://mlsanity.com/safety/watermarking-generated-content-and-models/)

## Builds on (most cited first, 3 of 3)

- [GPT detectors are biased against non-native English writers](https://mlsanity.com/paper/2304.02819/) (2023)
- [Testing of Detection Tools for AI-Generated Text](https://mlsanity.com/paper/2306.15666/) (2023)
- [Distributionally Robust Optimization: A Review](https://mlsanity.com/paper/1908.05659/) (2019)

## Built on by (most cited first, 30 of 57)

- [SoK: Watermarking for AI-Generated Content](https://mlsanity.com/paper/2411.18479/) (2024)
- [Theoretically Grounded Framework for LLM Watermarking: A Distribution-Adaptive Approach](https://mlsanity.com/paper/2410.02890/) (2024)
- [AdaDetectGPT: Adaptive Detection of LLM-Generated Text with Statistical Guarantees](https://mlsanity.com/paper/2510.01268/) (2025)
- [Debiasing Watermarks for Large Language Models via Maximal Coupling](https://mlsanity.com/paper/2411.11203/) (2024)
- [Watermarking Language Models for Many Adaptive Users](https://mlsanity.com/paper/2405.11109/) (2024)
- [WAPITI: A Watermark for Finetuned Open-Source LLMs](https://mlsanity.com/paper/2410.06467/) (2024)
- [Dataset Protection via Watermarked Canaries in Retrieval-Augmented LLMs](https://mlsanity.com/paper/2502.10673/) (2025)
- [Recursive Learning Without Collapse: A Weighting-Based Stabilization Framework](https://mlsanity.com/paper/2502.18049/) (2025)
- [Defending LLM Watermarking Against Spoofing Attacks with Contrastive Representation Learning](https://mlsanity.com/paper/2504.06575/) (2025)
- [In-Context Watermarks for Large Language Models](https://mlsanity.com/paper/2505.16934/) (2025)
- [Detecting LLM-Generated Text with Performance Guarantees](https://mlsanity.com/paper/2601.06586/) (2026)
- [Optimized Couplings for Watermarking Large Language Models](https://mlsanity.com/paper/2505.08878/) (2025)
- [Inevitable Trade-off between Watermark Strength and Speculative Sampling Efficiency for Language Models](https://mlsanity.com/paper/2410.20418/) (2024)
- [Who Taught You That? Tracing Teachers in Model Distillation](https://mlsanity.com/paper/2502.06659/) (2025)
- [Position: LLM Watermarking Should Align Stakeholders' Incentives for Practical Adoption](https://mlsanity.com/paper/2510.18333/) (2025)
- [PASA: A Principled Embedding-Space Watermarking Approach for LLM-Generated Text under Semantic-Invariant Attacks](https://mlsanity.com/paper/2605.10977/) (2026)
- [Do Large Language Models (Really) Need Statistical Foundations?](https://mlsanity.com/paper/2505.19145/) (2025)
- [On the Empirical Power of Goodness-of-Fit Tests in Watermark Detection](https://mlsanity.com/paper/2510.03944/) (2025)
- [Watermarking Discrete Diffusion Language Models](https://mlsanity.com/paper/2511.02083/) (2025)
- [A Unified Framework for LLM Watermarks](https://mlsanity.com/paper/2602.06754/) (2026)
- [Segmenting Watermarked Texts From Language Models](https://mlsanity.com/paper/2410.20670/) (2024)
- [SEFD: Semantic-Enhanced Framework for Detecting LLM-Generated Text](https://mlsanity.com/paper/2411.12764/) (2024)
- [A Statistical Hypothesis Testing Framework for Data Misappropriation Detection in Large Language Models](https://mlsanity.com/paper/2501.02441/) (2025)
- [Online LLM watermark detection via e-processes](https://mlsanity.com/paper/2602.14286/) (2026)
- [Refined Detection for Gumbel Watermarking](https://mlsanity.com/paper/2603.30017/) (2026)
- [Perturbation is All You Need for Extrapolating Language Models](https://mlsanity.com/paper/2605.04344/) (2026)
- [LLM Watermarking Using Mixtures and Statistical-to-Computational Gaps](https://mlsanity.com/paper/2505.01484/) (2025)
- [Towards Structurally Explainable Machine-Generated Text Detection: A Graph-Perspective Framework](https://mlsanity.com/paper/2505.12507/) (2025)
- [HeavyWater and SimplexWater: Distortion-Free LLM Watermarks for Low-Entropy Next-Token Predictions](https://mlsanity.com/paper/2506.06409/) (2025)
- [Robust Spectral Watermark for Synthetic Tabular Data](https://mlsanity.com/paper/2511.21600/) (2025)

## Abstract (arXiv)

Since ChatGPT was introduced in November 2022, embedding (nearly) unnoticeable statistical signals into text generated by large language models (LLMs), also known as watermarking, has been used as a principled approach to provable detection of LLM-generated text from its human-written counterpart. In this paper, we introduce a general and flexible framework for reasoning about the statistical efficiency of watermarks and designing powerful detection rules. Inspired by the hypothesis testing formulation of watermark detection, our framework starts by selecting a pivotal statistic of the text and a secret key -- provided by the LLM to the verifier -- to enable controlling the false positive rate (the error of mistakenly detecting human-written text as LLM-generated). Next, this framework allows one to evaluate the power of watermark detection rules by obtaining a closed-form expression of the asymptotic false negative rate (the error of incorrectly classifying LLM-generated text as human-written). Our framework further reduces the problem of determining the optimal detection rule to solving a minimax optimization program. We apply this framework to two representative watermarks -- one of which has been internally implemented at OpenAI -- and obtain several findings that can be instrumental in guiding the practice of implementing watermarks. In particular, we derive optimal detection rules for these watermarks under our framework. These theoretically derived detection rules are demonstrated to be competitive and sometimes enjoy a higher power than existing detection approaches through numerical experiments.