# Dataset Protection via Watermarked Canaries in Retrieval-Augmented LLMs

arXiv: https://arxiv.org/abs/2502.10673 · HTML: https://mlsanity.com/paper/2502.10673/

## Key facts

- Cited by 12 papers in the map (data through 14 Oct 2026).
- Placed in Privacy and security of RAG, one of 15 directions in Retrieval-augmented generation.
- Builds on 31 of its 55 references that are in the map.
- First posted on arXiv 15 Feb 2025.

## Placed in

- Retrieval-augmented generation / [Privacy and security of RAG](https://mlsanity.com/rag/privacy-and-security-of-rag/)
- LLM safety and security / [Watermarking generated content and models](https://mlsanity.com/safety/watermarking-generated-content-and-models/)

## Builds on (most cited first, 30 of 31)

- [Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks](https://mlsanity.com/paper/2005.11401/) (2020)
- [Dense Passage Retrieval for Open-Domain Question Answering](https://mlsanity.com/paper/2004.04906/) (2020)
- [MS MARCO: A Human Generated MAchine Reading COmprehension Dataset](https://mlsanity.com/paper/1611.09268/) (2016)
- [Unsupervised Dense Information Retrieval with Contrastive Learning](https://mlsanity.com/paper/2112.09118/) (2021)
- [DROP: A Reading Comprehension Benchmark Requiring Discrete Reasoning Over Paragraphs](https://mlsanity.com/paper/1903.00161/) (2019)
- [Approximate Nearest Neighbor Negative Contrastive Learning for Dense Text Retrieval](https://mlsanity.com/paper/2007.00808/) (2020)
- [MAUVE: Measuring the Gap Between Neural Text and Human Text using Divergence Frontiers](https://mlsanity.com/paper/2102.01454/) (2021)
- [AgentPoison: Red-teaming LLM Agents via Poisoning Memory or Knowledge Bases](https://mlsanity.com/paper/2407.12784/) (2024)
- [Robust Distortion-free Watermarks for Language Models](https://mlsanity.com/paper/2307.15593/) (2023)
- [Provable Robust Watermarking for AI-Generated Text](https://mlsanity.com/paper/2306.17439/) (2023)
- [QuRating: Selecting High-Quality Data for Training Language Models](https://mlsanity.com/paper/2402.09739/) (2024)
- [Dataset Inference: Ownership Resolution in Machine Learning](https://mlsanity.com/paper/2104.10706/) (2021)
- [A Semantic Invariant Robust Watermark for Large Language Models](https://mlsanity.com/paper/2310.06356/) (2023)
- [LLM Dataset Inference: Did you train on my dataset?](https://mlsanity.com/paper/2406.06443/) (2024)
- [MarkLLM: An Open-Source Toolkit for LLM Watermarking](https://mlsanity.com/paper/2405.10051/) (2024)
- [Follow My Instruction and Spill the Beans: Scalable Data Extraction from Retrieval-Augmented Generation Systems](https://mlsanity.com/paper/2402.17840/) (2024)
- [Domain Watermark: Effective and Harmless Dataset Copyright Protection is Closed at Hand](https://mlsanity.com/paper/2310.14942/) (2023)
- [A Statistical Framework of Watermarks for Large Language Models: Pivot, Detection Efficiency and Optimal Rules](https://mlsanity.com/paper/2404.01245/) (2024)
- [WaterMax: breaking the LLM watermark detectability-robustness-quality trade-off](https://mlsanity.com/paper/2403.04808/) (2024)
- [Image Watermarks are Removable Using Controllable Regeneration from Clean Noise](https://mlsanity.com/paper/2410.05470/) (2024)
- [Robustifying Safety-Aligned Large Language Models through Clean Data Curation](https://mlsanity.com/paper/2405.19358/) (2024)
- [GumbelSoft: Diversified Language Model Watermarking via the GumbelMax-trick](https://mlsanity.com/paper/2402.12948/) (2024)
- [Robust Detection of Watermarks for Large Language Models Under Human Edits](https://mlsanity.com/paper/2411.13868/) (2024)
- [TrojText: Test-time Invisible Textual Trojan Insertion](https://mlsanity.com/paper/2303.02242/) (2023)
- [Watermarking Text Data on Large Language Models for Dataset Copyright](https://mlsanity.com/paper/2305.13257/) (2023)
- [CPR: Retrieval Augmented Generation for Copyright Protection](https://mlsanity.com/paper/2403.18920/) (2024)
- [Can Watermarking Large Language Models Prevent Copyrighted Text Generation and Hide Training Data?](https://mlsanity.com/paper/2407.17417/) (2024)
- [Distortion-free Watermarks are not Truly Distortion-free under Watermark Key Collisions](https://mlsanity.com/paper/2406.02603/) (2024)
- [WaterSeeker: Pioneering Efficient Detection of Watermarked Segments in Large Documents](https://mlsanity.com/paper/2409.05112/) (2024)
- [Distributional Information Embedding: A Framework for Multi-bit Watermarking](https://mlsanity.com/paper/2501.16558/) (2025)

## Built on by (most cited first, 12 of 12)

- [MemMark: State-Evolution Attribution Watermarking for Agent Long-Term Memory Systems](https://mlsanity.com/paper/2605.25002/) (2026)
- [In-Context Watermarks for Large Language Models](https://mlsanity.com/paper/2505.16934/) (2025)
- [Position: LLM Watermarking Should Align Stakeholders' Incentives for Practical Adoption](https://mlsanity.com/paper/2510.18333/) (2025)
- [ImageSentinel: Protecting Visual Datasets from Unauthorized Retrieval-Augmented Image Generation](https://mlsanity.com/paper/2510.12119/) (2025)
- [Making Theft Useless: Adulteration-Based Protection of Proprietary Knowledge Graphs in GraphRAG Systems](https://mlsanity.com/paper/2601.00274/) (2026)
- [Boosting Adversarial Transferability via Commonality-Oriented Gradient Optimization](https://mlsanity.com/paper/2506.06992/) (2025)
- [Safeguarding Multimodal Knowledge Copyright in the RAG-as-a-Service Environment](https://mlsanity.com/paper/2506.10030/) (2025)
- [TokenPure: Watermark Removal through Tokenized Appearance and Structural Guidance](https://mlsanity.com/paper/2512.01314/) (2025)
- [Detecting RAG Extraction Attack via Dual-Path Runtime Integrity Game](https://mlsanity.com/paper/2604.10717/) (2026)
- [SentinelRAG: Synthetic Sentinel Knowledge for RAG Database Copyright Protection](https://mlsanity.com/paper/2606.05787/) (2026)
- [Learning to Follow In-Context Watermark Instructions via Self-Distillation](https://mlsanity.com/paper/2608.29030/) (2026)
- [Rent-a-RAG: Embedding-Space Watermarks for Auditing Third-Party RAG](https://mlsanity.com/paper/2609.03749/) (2026)

## Abstract (arXiv)

Retrieval-Augmented Generation (RAG) has become an effective method for enhancing large language models (LLMs) with up-to-date knowledge. However, it may pose a risk of copyright infringement, as IP datasets may be incorporated into the knowledge database by malicious Retrieval-Augmented LLMs (RA-LLMs) without authorization. To protect the rights of the dataset owner, an effective dataset membership inference algorithm for RA-LLMs is needed. In this work, we introduce a novel approach, textitCanaryTrace, to safeguard the ownership of text datasets and effectively detect unauthorized use by RA-LLMs. Our approach preserves the original data completely unchanged while protecting it by inserting specifically designed canary documents into the IP dataset. These canary documents are created with synthetic content and embedded watermarks to ensure uniqueness, consistency, and statistical provability. During the detection process, unauthorized usage is identified by querying the canary documents and analyzing the responses of RA-LLMs for statistical evidence of the embedded watermark. Our experimental results demonstrate high query efficiency, detectability, and consistency, along with minimal perturbation to the original dataset, all without compromising the performance of the RAG system.