# Optimizing Adaptive Attacks against Watermarks for Language Models

arXiv: https://arxiv.org/abs/2410.02440 · HTML: https://mlsanity.com/paper/2410.02440/

## Key facts

- Cited by 16 papers in the map (data through 14 Oct 2026).
- Placed in Watermarking generated content and models, one of 12 directions in LLM safety and security.
- Builds on 27 of its 37 references that are in the map.
- First posted on arXiv 3 Oct 2024.

## Placed in

- LLM safety and security / [Watermarking generated content and models](https://mlsanity.com/safety/watermarking-generated-content-and-models/)

## Builds on (most cited first, 27 of 27)

- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [The Llama 3 Herd of Models](https://mlsanity.com/paper/2407.21783/) (2024)
- [Chain-of-Thought Prompting Elicits Reasoning in Large Language Models](https://mlsanity.com/paper/2201.11903/) (2022)
- [Direct Preference Optimization: Your Language Model is Secretly a Reward Model](https://mlsanity.com/paper/2305.18290/) (2023)
- [Efficient Memory Management for Large Language Model Serving with PagedAttention](https://mlsanity.com/paper/2309.06180/) (2023)
- [Ethical and social risks of harm from Language Models](https://mlsanity.com/paper/2112.04359/) (2021)
- [A Watermark for Large Language Models](https://mlsanity.com/paper/2301.10226/) (2023)
- [MAUVE: Measuring the Gap Between Neural Text and Human Text using Divergence Frontiers](https://mlsanity.com/paper/2102.01454/) (2021)
- [Paraphrasing evades detectors of AI-generated text, but retrieval is an effective defense](https://mlsanity.com/paper/2303.13408/) (2023)
- [Robust Distortion-free Watermarks for Language Models](https://mlsanity.com/paper/2307.15593/) (2023)
- [Provable Robust Watermarking for AI-Generated Text](https://mlsanity.com/paper/2306.17439/) (2023)
- [On the Reliability of Watermarks for Large Language Models](https://mlsanity.com/paper/2306.04634/) (2023)
- [A Semantic Invariant Robust Watermark for Large Language Models](https://mlsanity.com/paper/2310.06356/) (2023)
- [Combating Misinformation in the Age of LLMs: Opportunities and Challenges](https://mlsanity.com/paper/2311.05656/) (2023)
- [MarkLLM: An Open-Source Toolkit for LLM Watermarking](https://mlsanity.com/paper/2405.10051/) (2024)
- [Watermark Stealing in Large Language Models](https://mlsanity.com/paper/2402.19361/) (2024)
- [Identifying and Mitigating the Security Risks of Generative AI](https://mlsanity.com/paper/2308.14840/) (2023)
- [Watermarks in the Sand: Impossibility of Strong Watermarking for Generative Models](https://mlsanity.com/paper/2311.04378/) (2023)
- [Evading Watermark based Detection of AI-Generated Content](https://mlsanity.com/paper/2305.03807/) (2023)
- [Proactive Detection of Voice Cloning with Localized Watermarking](https://mlsanity.com/paper/2401.17264/) (2024)
- [Mark My Words: Analyzing and Evaluating Language Model Watermarks](https://mlsanity.com/paper/2312.00273/) (2023)
- [Leveraging Optimization for Adaptive Attacks on Image Watermarks](https://mlsanity.com/paper/2309.16952/) (2023)
- [No Free Lunch in LLM Watermarking: Trade-offs in Watermarking Design Choices](https://mlsanity.com/paper/2402.16187/) (2024)
- [PTW: Pivotal Tuning Watermarking for Pre-Trained Image Generators](https://mlsanity.com/paper/2304.07361/) (2023)
- [The Ethical Need for Watermarks in Machine-Generated Language](https://mlsanity.com/paper/2209.03118/) (2022)
- [Deepfakes, Misinformation, and Disinformation in the Era of Frontier AI, Generative AI, and Large AI Models](https://mlsanity.com/paper/2311.17394/) (2023)
- [Stable Signature is Unstable: Removing Image Watermark from Diffusion Models](https://mlsanity.com/paper/2405.07145/) (2024)

## Built on by (most cited first, 16 of 16)

- [Sandcastles in the Storm: Revisiting the (Im)possibility of Strong Watermarking](https://mlsanity.com/paper/2505.06827/) (2025)
- [Mitigating Watermark Forgery in Generative Models via Randomized Key Selection](https://mlsanity.com/paper/2507.07871/) (2025)
- [RLCracker: Evaluating the Worst-Case Vulnerability of LLM Watermarks with Adaptive RL Attacks](https://mlsanity.com/paper/2509.20924/) (2025)
- [RLSpoofer: A Lightweight Evaluator for LLM Watermark Spoofing Resilience](https://mlsanity.com/paper/2604.11546/) (2026)
- [Watermarking Should Be Treated as a Monitoring Primitive](https://mlsanity.com/paper/2605.13095/) (2026)
- [Bridging Today and the Future of Humanity: AI Safety in 2024 and Beyond](https://mlsanity.com/paper/2410.18114/) (2024)
- [Latent-Mark: An Audio Watermark Robust to Neural Codec Compression](https://mlsanity.com/paper/2603.05310/) (2026)
- [Adaptively Robust LLM Monitoring via Activation Watermarking](https://mlsanity.com/paper/2603.23171/) (2026)
- [Semantic Watermarking with Order-Robust Detection over Sub-sentence Units](https://mlsanity.com/paper/2608.27666/) (2026)
- [TimeMark: A Trustworthy Time Watermarking Framework for Exact Generation-Time Recovery from AIGC](https://mlsanity.com/paper/2604.12216/) (2026)
- [Stability-Aware Feature Design for Robust Watermark Detection in Machine-Generated Text](https://mlsanity.com/paper/2608.18102/) (2026)
- [PURA: Provably Unbiased and Robust Multi-Bit Watermarking for AI-Generated Text Attribution](https://mlsanity.com/paper/2608.22218/) (2026)
- [GenoTrace: Inheritable Watermarks for Genome Foundation Model Distillation](https://mlsanity.com/paper/2609.35881/) (2026)
- [Invisible Ink, Visible Lies: How Production Watermarking Causes LLMs to Hallucinate](https://mlsanity.com/paper/2610.04860/) (2026)
- [Certification of Real Images through Calibrated Content Authentication](https://mlsanity.com/paper/2610.05870/) (2026)
- [Could LLM Watermark Detection be Public?](https://mlsanity.com/paper/2610.12106/) (2026)

## Abstract (arXiv)

Large Language Models (LLMs) can be misused to spread unwanted content at scale. Content watermarking deters misuse by hiding messages in content, enabling its detection using a secret watermarking key. Robustness is a core security property, stating that evading detection requires (significant) degradation of the content's quality. Many LLM watermarking methods have been proposed, but robustness is tested only against non-adaptive attackers who lack knowledge of the watermarking method and can find only suboptimal attacks. We formulate watermark robustness as an objective function and use preference-based optimization to tune adaptive attacks against the specific watermarking method. Our evaluation shows that (i) adaptive attacks evade detection against all surveyed watermarks, (ii) training against any watermark succeeds in evading unseen watermarks, and (iii) optimization-based attacks are cost-effective. Our findings underscore the need to test robustness against adaptively tuned attacks. We release our adaptively optimized paraphrasers at https://github.com/nilslukas/ada-wm-evasion.