# Duwak: Dual Watermarks in Large Language Models

arXiv: https://arxiv.org/abs/2403.13000 · HTML: https://mlsanity.com/paper/2403.13000/

## Key facts

- Cited by 11 papers in the map (data through 14 Oct 2026).
- Placed in Watermarking generated content and models, one of 12 directions in LLM safety and security.
- Builds on 21 of its 44 references that are in the map.
- First posted on arXiv 12 Mar 2024.

## Placed in

- LLM safety and security / [Watermarking generated content and models](https://mlsanity.com/safety/watermarking-generated-content-and-models/)

## Builds on (most cited first, 21 of 21)

- [Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena](https://mlsanity.com/paper/2306.05685/) (2023)
- [The Curious Case of Neural Text Degeneration](https://mlsanity.com/paper/1904.09751/) (2019)
- [Code Llama: Open Foundation Models for Code](https://mlsanity.com/paper/2308.12950/) (2023)
- [Hierarchical Neural Story Generation](https://mlsanity.com/paper/1805.04833/) (2018)
- [How Contextual are Contextualized Word Representations? Comparing the Geometry of BERT, ELMo, and GPT-2 Embeddings](https://mlsanity.com/paper/1909.00512/) (2019)
- [DetectGPT: Zero-Shot Machine-Generated Text Detection using Probability Curvature](https://mlsanity.com/paper/2301.11305/) (2023)
- [Release Strategies and the Social Impacts of Language Models](https://mlsanity.com/paper/1908.09203/) (2019)
- [Neural Text Generation with Unlikelihood Training](https://mlsanity.com/paper/1908.04319/) (2019)
- [GLTR: Statistical Detection and Visualization of Generated Text](https://mlsanity.com/paper/1906.04043/) (2019)
- [MAUVE: Measuring the Gap Between Neural Text and Human Text using Divergence Frontiers](https://mlsanity.com/paper/2102.01454/) (2021)
- [Large Language Models Are State-of-the-Art Evaluators of Translation Quality](https://mlsanity.com/paper/2302.14520/) (2023)
- [Robust Distortion-free Watermarks for Language Models](https://mlsanity.com/paper/2307.15593/) (2023)
- [Provable Robust Watermarking for AI-Generated Text](https://mlsanity.com/paper/2306.17439/) (2023)
- [Prompting Large Language Model for Machine Translation: A Case Study](https://mlsanity.com/paper/2301.07069/) (2023)
- [A Contrastive Framework for Neural Text Generation](https://mlsanity.com/paper/2202.06417/) (2022)
- [RADAR: Robust AI-Text Detection via Adversarial Learning](https://mlsanity.com/paper/2307.03838/) (2023)
- [Who Wrote this Code? Watermarking for Code Generation](https://mlsanity.com/paper/2305.15060/) (2023)
- [Demystifying the Draft EU Artificial Intelligence Act](https://mlsanity.com/paper/2107.03721/) (2021)
- [A Robust Semantics-based Watermark for Large Language Model against Paraphrasing](https://mlsanity.com/paper/2311.08721/) (2023)
- [Contrastive Search Is What You Need For Neural Text Generation](https://mlsanity.com/paper/2210.14140/) (2022)
- [Smaller Language Models are Better Black-box Machine-Generated Text Detectors](https://mlsanity.com/paper/2305.09859/) (2023)

## Built on by (most cited first, 11 of 11)

- [Watermarking Techniques for Large Language Models: A Survey](https://mlsanity.com/paper/2409.00089/) (2024)
- [In-Context Watermarks for Large Language Models](https://mlsanity.com/paper/2505.16934/) (2025)
- [A Survey on Responsible Generative AI: What to Generate and What Not](https://mlsanity.com/paper/2404.05783/) (2024)
- [Position: LLM Watermarking Should Align Stakeholders' Incentives for Practical Adoption](https://mlsanity.com/paper/2510.18333/) (2025)
- [TimeWak: Temporal Chained-Hashing Watermark for Time Series Data](https://mlsanity.com/paper/2506.06407/) (2025)
- [Enhancing Watermarking Quality for LLMs via Contextual Generation States Awareness](https://mlsanity.com/paper/2506.07403/) (2025)
- [SoK: Are Watermarks in LLMs Ready for Deployment?](https://mlsanity.com/paper/2506.05594/) (2025)
- [LLM-Text Watermarking based on Lagrange Interpolation](https://mlsanity.com/paper/2505.05712/) (2025)
- [Optimal Multi-bit Generative Watermarking Schemes Under Worst-Case False-Alarm Constraints](https://mlsanity.com/paper/2604.08759/) (2026)
- [Synchronization-Free Algebraic Fingerprints for Large Language Models: From Autoregressive to Diffusion Models](https://mlsanity.com/paper/2607.16648/) (2026)
- [Learning to Follow In-Context Watermark Instructions via Self-Distillation](https://mlsanity.com/paper/2608.29030/) (2026)

## Abstract (arXiv)

As large language models (LLM) are increasingly used for text generation tasks, it is critical to audit their usages, govern their applications, and mitigate their potential harms. Existing watermark techniques are shown effective in embedding single human-imperceptible and machine-detectable patterns without significantly affecting generated text quality and semantics. However, the efficiency in detecting watermarks, i.e., the minimum number of tokens required to assert detection with significance and robustness against post-editing, is still debatable. In this paper, we propose, Duwak, to fundamentally enhance the efficiency and quality of watermarking by embedding dual secret patterns in both token probability distribution and sampling schemes. To mitigate expression degradation caused by biasing toward certain tokens, we design a contrastive search to watermark the sampling scheme, which minimizes the token repetition and enhances the diversity. We theoretically explain the interdependency of the two watermarks within Duwak. We evaluate Duwak extensively on Llama2 under various post-editing attacks, against four state-of-the-art watermarking techniques and combinations of them. Our results show that Duwak marked text achieves the highest watermarked text quality at the lowest required token count for detection, up to 70% tokens less than existing approaches, especially under post paraphrasing.