# Bypassing LLM Watermarks with Color-Aware Substitutions

arXiv: https://arxiv.org/abs/2403.14719 · HTML: https://mlsanity.com/paper/2403.14719/

## Key facts

- Cited by 28 papers in the map (data through 14 Oct 2026).
- Placed in Watermarking generated content and models, one of 12 directions in LLM safety and security.
- Builds on 16 of its 36 references that are in the map.
- First posted on arXiv 19 Mar 2024.

## Placed in

- LLM safety and security / [Watermarking generated content and models](https://mlsanity.com/safety/watermarking-generated-content-and-models/)

## Builds on (most cited first, 16 of 16)

- [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://mlsanity.com/paper/1910.10683/) (2019)
- [Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena](https://mlsanity.com/paper/2306.05685/) (2023)
- [DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lighter](https://mlsanity.com/paper/1910.01108/) (2019)
- [Paraphrasing evades detectors of AI-generated text, but retrieval is an effective defense](https://mlsanity.com/paper/2303.13408/) (2023)
- [Can AI-Generated Text be Reliably Detected?](https://mlsanity.com/paper/2303.11156/) (2023)
- [Robust Distortion-free Watermarks for Language Models](https://mlsanity.com/paper/2307.15593/) (2023)
- [Provable Robust Watermarking for AI-Generated Text](https://mlsanity.com/paper/2306.17439/) (2023)
- [Can LLM-Generated Misinformation Be Detected?](https://mlsanity.com/paper/2309.13788/) (2023)
- [Watermarks in the Sand: Impossibility of Strong Watermarking for Generative Models](https://mlsanity.com/paper/2311.04378/) (2023)
- [Protecting Intellectual Property of Language Generation APIs with Lexical Watermark](https://mlsanity.com/paper/2112.02701/) (2021)
- [CATER: Intellectual Property Protection on Text Generation APIs via Conditional Watermarks](https://mlsanity.com/paper/2209.08773/) (2022)
- [A Survey on Detection of LLMs-Generated Content](https://mlsanity.com/paper/2310.15654/) (2023)
- [Red Teaming Language Model Detectors with Language Models](https://mlsanity.com/paper/2305.19713/) (2023)
- [Large Language Models can be Guided to Evade AI-Generated Text Detection](https://mlsanity.com/paper/2305.10847/) (2023)
- [Towards Possibilities & Impossibilities of AI-generated Text Detection: A Survey](https://mlsanity.com/paper/2310.15264/) (2023)
- [Protecting Your NLG Models with Semantic and Robust Watermarks](https://mlsanity.com/paper/2112.05428/) (2021)

## Built on by (most cited first, 28 of 28)

- [Can LLM Watermarks Robustly Prevent Unauthorized Knowledge Distillation?](https://mlsanity.com/paper/2502.11598/) (2025)
- [Revealing Weaknesses in Text Watermarking Through Self-Information Rewrite Attacks](https://mlsanity.com/paper/2505.05190/) (2025)
- [A Survey of Attacks on Large Language Models](https://mlsanity.com/paper/2505.12567/) (2025)
- [Large Language Model Watermark Stealing With Mixed Integer Programming](https://mlsanity.com/paper/2405.19677/) (2024)
- [SoK: On the Role and Future of AIGC Watermarking in the Era of Gen-AI](https://mlsanity.com/paper/2411.11478/) (2024)
- [Can Watermarked LLMs be Identified by Users via Crafted Prompts?](https://mlsanity.com/paper/2410.03168/) (2024)
- [Enhancing LLM Watermark Resilience Against Both Scrubbing and Spoofing Attacks](https://mlsanity.com/paper/2507.06274/) (2025)
- [Character-Level Perturbations Disrupt LLM Watermarks](https://mlsanity.com/paper/2509.09112/) (2025)
- [Are Robust LLM Fingerprints Adversarially Robust?](https://mlsanity.com/paper/2509.26598/) (2025)
- [Discovering Spoofing Attempts on Language Model Watermarks](https://mlsanity.com/paper/2410.02693/) (2024)
- [Humanizing the Machine: Proxy Attacks to Mislead LLM Detectors](https://mlsanity.com/paper/2410.19230/) (2024)
- [Survey on Plagiarism Detection in Large Language Models: The Impact of ChatGPT and Gemini on Academic Integrity](https://mlsanity.com/paper/2407.13105/) (2024)
- [From Trade-off to Synergy: A Versatile Symbiotic Watermarking Framework for Large Language Models](https://mlsanity.com/paper/2505.09924/) (2025)
- [Mitigating Watermark Forgery in Generative Models via Randomized Key Selection](https://mlsanity.com/paper/2507.07871/) (2025)
- [PhantomHunter: Detecting Unseen Privately-Tuned LLM-Generated Text via Family-Aware Learning](https://mlsanity.com/paper/2506.15683/) (2025)
- [A Visual Semantic Adaptive Watermark grounded by Prefix-Tuning for Large Vision-Language Model](https://mlsanity.com/paper/2601.07291/) (2026)
- [Watermarking Should Be Treated as a Monitoring Primitive](https://mlsanity.com/paper/2605.13095/) (2026)
- [LLM Watermark Evasion via Bias Inversion](https://mlsanity.com/paper/2509.23019/) (2025)
- [AGMark: Attention-Guided Dynamic Watermarking for Large Vision-Language Models](https://mlsanity.com/paper/2602.09611/) (2026)
- [Robust LLM Watermarking with Minimal Semantic Distortion for IP Protection](https://mlsanity.com/paper/2605.23175/) (2026)
- [Linear Ensembles Wash Away Watermarks: On the Fragility of Distributional Perturbations in LLMs](https://mlsanity.com/paper/2605.30501/) (2026)
- [Breaking Distortion-free Watermarks in Large Language Models](https://mlsanity.com/paper/2502.18608/) (2025)
- [Blind PRNG Hijacking: An Undetectable Integrity-Preserving Attack Against LLM Watermarking](https://mlsanity.com/paper/2605.28632/) (2026)
- [DHMark: Public-Key Watermarking for LLM-Generated Text via Diffie-Hellman-Guided Rejection Sampling](https://mlsanity.com/paper/2608.03093/) (2026)
- [TripPattern: A Pattern-based Text Watermarking Method for Large Language Models](https://mlsanity.com/paper/2609.12472/) (2026)
- [TANGO: Watermarking Masked Diffusion Language Models in Token Pairs](https://mlsanity.com/paper/2609.35224/) (2026)
- [Invisible Ink, Visible Lies: How Production Watermarking Causes LLMs to Hallucinate](https://mlsanity.com/paper/2610.04860/) (2026)
- [Pin It on Anyone: Certified Targeted Displacement of Multi-Bit LLM Watermarks Without the Key](https://mlsanity.com/paper/2610.18359/) (2026)

## Abstract (arXiv)

Watermarking approaches are proposed to identify if text being circulated is human or large language model (LLM) generated. The state-of-the-art watermarking strategy of Kirchenbauer et al. (2023a) biases the LLM to generate specific (``green'') tokens. However, determining the robustness of this watermarking method is an open problem. Existing attack methods fail to evade detection for longer text segments. We overcome this limitation, and propose em Self Color Testing-based Substitution (SCTS), the first ``color-aware'' attack. SCTS obtains color information by strategically prompting the watermarked LLM and comparing output tokens frequencies. It uses this information to determine token colors, and substitutes green tokens with non-green ones. In our experiments, SCTS successfully evades watermark detection using fewer number of edits than related work. Additionally, we show both theoretically and empirically that SCTS can remove the watermark for arbitrarily long watermarked text.