# Black-Box Detection of Language Model Watermarks

arXiv: https://arxiv.org/abs/2405.20777 · HTML: https://mlsanity.com/paper/2405.20777/

## Key facts

- Cited by 23 papers in the map (data through 14 Oct 2026).
- Placed in Watermarking generated content and models, one of 12 directions in LLM safety and security.
- Builds on 17 of its 28 references that are in the map.
- First posted on arXiv 28 May 2024.

## Placed in

- LLM safety and security / [Watermarking generated content and models](https://mlsanity.com/safety/watermarking-generated-content-and-models/)

## Builds on (most cited first, 17 of 17)

- [On the Opportunities and Risks of Foundation Models](https://mlsanity.com/paper/2108.07258/) (2021)
- [A Watermark for Large Language Models](https://mlsanity.com/paper/2301.10226/) (2023)
- [Can AI-Generated Text be Reliably Detected?](https://mlsanity.com/paper/2303.11156/) (2023)
- [Robust Distortion-free Watermarks for Language Models](https://mlsanity.com/paper/2307.15593/) (2023)
- [Undetectable Watermarks for Language Models](https://mlsanity.com/paper/2306.09194/) (2023)
- [On the Reliability of Watermarks for Large Language Models](https://mlsanity.com/paper/2306.04634/) (2023)
- [Stealing Part of a Production Language Model](https://mlsanity.com/paper/2403.06634/) (2024)
- [A Semantic Invariant Robust Watermark for Large Language Models](https://mlsanity.com/paper/2310.06356/) (2023)
- [Unbiased Watermark for Large Language Models](https://mlsanity.com/paper/2310.10669/) (2023)
- [Watermark Stealing in Large Language Models](https://mlsanity.com/paper/2402.19361/) (2024)
- [A Robust Semantics-based Watermark for Large Language Model against Paraphrasing](https://mlsanity.com/paper/2311.08721/) (2023)
- [On the Learnability of Watermarks for Language Models](https://mlsanity.com/paper/2312.04469/) (2023)
- [Publicly-Detectable Watermarking for Language Models](https://mlsanity.com/paper/2310.18491/) (2023)
- [Mark My Words: Analyzing and Evaluating Language Model Watermarks](https://mlsanity.com/paper/2312.00273/) (2023)
- [A Resilient and Accessible Distribution-Preserving Watermark for Large Language Models](https://mlsanity.com/paper/2310.07710/) (2023)
- [Stealing the Decoding Algorithms of Language Models](https://mlsanity.com/paper/2303.04729/) (2023)
- [Baselines for Identifying Watermarked Large Language Models](https://mlsanity.com/paper/2305.18456/) (2023)

## Built on by (most cited first, 23 of 23)

- [A Watermark for Black-Box Language Models](https://mlsanity.com/paper/2410.02099/) (2024)
- [De-mark: Watermark Removal in Large Language Models](https://mlsanity.com/paper/2410.13808/) (2024)
- [Can Watermarked LLMs be Identified by Users via Crafted Prompts?](https://mlsanity.com/paper/2410.03168/) (2024)
- [Towards Watermarking of Open-Source LLMs](https://mlsanity.com/paper/2502.10525/) (2025)
- [Watermarking Diffusion Language Models](https://mlsanity.com/paper/2509.24368/) (2025)
- [LLM Fingerprinting via Semantically Conditioned Watermarks](https://mlsanity.com/paper/2505.16723/) (2025)
- [Human Texts Are Outliers: Detecting LLM-generated Texts via Out-of-distribution Detection](https://mlsanity.com/paper/2510.08602/) (2025)
- [SAEMark: Steering Personalized Multilingual LLM Watermarks with Sparse Autoencoders](https://mlsanity.com/paper/2508.08211/) (2025)
- [Position: LLM Watermarking Should Align Stakeholders' Incentives for Practical Adoption](https://mlsanity.com/paper/2510.18333/) (2025)
- [Discovering Spoofing Attempts on Language Model Watermarks](https://mlsanity.com/paper/2410.02693/) (2024)
- [Diversity Boosts AI-Generated Text Detection](https://mlsanity.com/paper/2509.18880/) (2025)
- [A Unified Framework for LLM Watermarks](https://mlsanity.com/paper/2602.06754/) (2026)
- [Revisiting Black-Box Model Ownership Verification through Information Theory](https://mlsanity.com/paper/2409.06130/) (2024)
- [Watermarking Should Be Treated as a Monitoring Primitive](https://mlsanity.com/paper/2605.13095/) (2026)
- [Safeguarding Multimodal Knowledge Copyright in the RAG-as-a-Service Environment](https://mlsanity.com/paper/2506.10030/) (2025)
- [SearchLLM: Detecting LLM Paraphrased Text by Measuring the Similarity with Regeneration of the Candidate Source via Search Engine](https://mlsanity.com/paper/2601.16512/) (2026)
- [Global Sketch-Based Watermarking for Diffusion Language Models](https://mlsanity.com/paper/2606.04486/) (2026)
- [WaterMoE: Expert-Routing-based Watermarking for High Fidelity and Efficiency](https://mlsanity.com/paper/2607.13099/) (2026)
- [Breaking Distortion-free Watermarks in Large Language Models](https://mlsanity.com/paper/2502.18608/) (2025)
- [Local Differences, Global Lessons: Insights from Organisation Policies for International Legislation](https://mlsanity.com/paper/2503.05737/) (2025)
- [On the Salience of Low-Probability Tokens for AI-Generated Text Detection: A Multiscale Uncertainty Perspective](https://mlsanity.com/paper/2606.02158/) (2026)
- [Rent-a-RAG: Embedding-Space Watermarks for Auditing Third-Party RAG](https://mlsanity.com/paper/2609.03749/) (2026)
- [Pin It on Anyone: Certified Targeted Displacement of Multi-Bit LLM Watermarks Without the Key](https://mlsanity.com/paper/2610.18359/) (2026)

## Abstract (arXiv)

Watermarking has emerged as a promising way to detect LLM-generated text, by augmenting LLM generations with later detectable signals. Recent work has proposed multiple families of watermarking schemes, several of which focus on preserving the LLM distribution. This distribution-preservation property is motivated by the fact that it is a tractable proxy for retaining LLM capabilities, as well as the inherently implied undetectability of the watermark by downstream users. Yet, despite much discourse around undetectability, no prior work has investigated the practical detectability of any of the current watermarking schemes in a realistic black-box setting. In this work we tackle this for the first time, developing rigorous statistical tests to detect the presence, and estimate parameters, of all three popular watermarking scheme families, using only a limited number of black-box queries. We experimentally confirm the effectiveness of our methods on a range of schemes and a diverse set of open-source models. Further, we validate the feasibility of our tests on real-world APIs. Our findings indicate that current watermarking schemes are more detectable than previously believed.