# Protecting Your NLG Models with Semantic and Robust Watermarks

arXiv: https://arxiv.org/abs/2112.05428 · HTML: https://mlsanity.com/paper/2112.05428/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 10 Dec 2021.

## Built on by (most cited first, 7 of 7)

- [Watermarking Pre-trained Language Models with Backdooring](https://mlsanity.com/paper/2210.07543/) (2022)
- [HuRef: HUman-REadable Fingerprint for Large Language Models](https://mlsanity.com/paper/2312.04828/) (2023)
- [Bypassing LLM Watermarks with Color-Aware Substitutions](https://mlsanity.com/paper/2403.14719/) (2024)
- [Deconstructing The Ethics of Large Language Models from Long-standing Issues to New-emerging Dilemmas: A Survey](https://mlsanity.com/paper/2406.05392/) (2024)
- [Deep Intellectual Property Protection: A Survey](https://mlsanity.com/paper/2304.14613/) (2023)
- [Taylor Unswift: Secured Weight Release for Large Language Models via Taylor Expansion](https://mlsanity.com/paper/2410.05331/) (2024)
- [Task-Agnostic Language Model Watermarking via High Entropy Passthrough Layers](https://mlsanity.com/paper/2412.12563/) (2024)

## Abstract (arXiv)

Natural language generation (NLG) applications have gained great popularity due to the powerful deep learning techniques and large training corpus. The deployed NLG models may be stolen or used without authorization, while watermarking has become a useful tool to protect Intellectual Property (IP) of deep models. However, existing watermarking technologies using backdoors are easily detected or harmful for NLG applications. In this paper, we propose a semantic and robust watermarking scheme for NLG models that utilize unharmful phrase pairs as watermarks for IP protection. The watermarks give NLG models personal preference for some special phrase combinations. Specifically, we generate watermarks by following a semantic combination pattern and systematically augment the watermark corpus to enhance the robustness. Then, we embed these watermarks into an NLG model without misleading its original attention mechanism. We conduct extensive experiments and the results demonstrate the effectiveness, robustness, and undetectability of the proposed scheme.