mlsanityData to 7 Oct 2026

Bypassing LLM Watermarks with Color-Aware Substitutions

arXiv 2403.14719 ↗First posted 19 Mar 2024cs.CR cs.CV cs.LGcited by 24

On the map: LLM safety and security → Watermarking generated content and models

Abstract (arXiv)

Watermarking approaches are proposed to identify if text being circulated is human or large language model (LLM) generated. The state-of-the-art watermarking strategy of Kirchenbauer et al. (2023a) biases the LLM to generate specific (``green'') tokens. However, determining the robustness of this watermarking method is an open problem. Existing attack methods fail to evade detection for longer text segments. We overcome this limitation, and propose em Self Color Testing-based Substitution (SCTS), the first ``color-aware'' attack. SCTS obtains color information by strategically prompting the watermarked LLM and comparing output tokens frequencies. It uses this information to determine token colors, and substitutes green tokens with non-green ones. In our experiments, SCTS successfully evades watermark detection using fewer number of edits than related work. Additionally, we show both theoretically and empirically that SCTS can remove the watermark for arbitrarily long watermarked text.

Key facts. Cited by 24 papers in the map (data through 7 Oct 2026). Placed in Watermarking generated content and models, one of 12 directions in LLM safety and security. Builds on 16 of its 36 references that are in the map. First posted on arXiv 19 Mar 2024.

Builds on

16 of 36 references.

Closest ancestors · 7

Same direction, newest first

  1. 2023Watermarks in the Sand: Impossibility of Strong Watermarking for Generative Models7 Nov
  2. 2023Robust Distortion-free Watermarks for Language Models28 Jul
  3. 2023Provable Robust Watermarking for AI-Generated Text30 Jun
  4. 2023Paraphrasing evades detectors of AI-generated text, but retrieval is an effective defense23 Mar
  5. 2023Can AI-Generated Text be Reliably Detected?17 Mar
  6. 2022CATER: Intellectual Property Protection on Text Generation APIs via Conditional Watermarks19 Sep
  7. 2021Protecting Intellectual Property of Language Generation APIs with Lexical Watermark5 Dec

Other references · 9

  1. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
  2. 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,393
  3. 2019DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lightercited by 4,243
  4. 2023Can LLM-Generated Misinformation Be Detected?cited by 173
  5. 2023A Survey on Detection of LLMs-Generated Contentcited by 70
  6. 2023Red Teaming Language Model Detectors with Language Modelscited by 55
  7. 2023Large Language Models can be Guided to Evade AI-Generated Text Detectioncited by 43
  8. 2023Towards Possibilities & Impossibilities of AI-generated Text Detection: A Surveycited by 28
  9. 2021Protecting Your NLG Models with Semantic and Robust Watermarkscited by 7

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 5; 2025: 11; 2026: 8.

24 papers.

Sort:

By area:

  1. 2025Can LLM Watermarks Robustly Prevent Unauthorized Knowledge Distillation?cited by 16
  2. 2024Large Language Model Watermark Stealing With Mixed Integer Programmingcited by 13
  3. 2025A Survey of Attacks on Large Language Modelscited by 13
  4. 2025Revealing Weaknesses in Text Watermarking Through Self-Information Rewrite Attackscited by 12
  5. 2024Can Watermarked LLMs be Identified by Users via Crafted Prompts?cited by 10
  6. 2025Enhancing LLM Watermark Resilience Against Both Scrubbing and Spoofing Attackscited by 8
  7. 2025Character-Level Perturbations Disrupt LLM Watermarkscited by 7
  8. 2025Are Robust LLM Fingerprints Adversarially Robust?cited by 7
  9. 2024Humanizing the Machine: Proxy Attacks to Mislead LLM Detectorscited by 6
  10. 2024Survey on Plagiarism Detection in Large Language Models: The Impact of ChatGPT and Gemini on Academic Integritycited by 5
  11. 2025From Trade-off to Synergy: A Versatile Symbiotic Watermarking Framework for Large Language Modelscited by 5
  12. 2024Discovering Spoofing Attempts on Language Model Watermarkscited by 4
12 more
  1. 2025Mitigating Watermark Forgery in Generative Models via Randomized Key Selectioncited by 4
  2. 2026A Visual Semantic Adaptive Watermark grounded by Prefix-Tuning for Large Vision-Language Modelcited by 3
  3. 2025PhantomHunter: Detecting Unseen Privately-Tuned LLM-Generated Text via Family-Aware Learningcited by 2
  4. 2026Watermarking Should Be Treated as a Monitoring Primitivecited by 2
  5. 2026AGMark: Attention-Guided Dynamic Watermarking for Large Vision-Language Modelscited by 1
  6. 2026Robust LLM Watermarking with Minimal Semantic Distortion for IP Protectioncited by 1
  7. 2025Breaking Distortion-free Watermarks in Large Language Modelscited by 0
  8. 2025LLM Watermark Evasion via Bias Inversioncited by 0
  9. 2026Blind PRNG Hijacking: An Undetectable Integrity-Preserving Attack Against LLM Watermarkingcited by 0
  10. 2026Linear Ensembles Wash Away Watermarks: On the Fragility of Distributional Perturbations in LLMscited by 0
  11. 2026TripPattern: A Pattern-based Text Watermarking Method for Large Language Modelscited by 0
  12. 2026TANGO: Watermarking Masked Diffusion Language Models in Token Pairscited by 0