# CPR: Retrieval Augmented Generation for Copyright Protection

arXiv: https://arxiv.org/abs/2403.18920 · HTML: https://mlsanity.com/paper/2403.18920/

## Key facts

- Cited by 24 papers in the map (data through 14 Oct 2026).
- Placed in Privacy and security of RAG, one of 15 directions in Retrieval-augmented generation.
- Builds on 57 of its 71 references that are in the map.
- First posted on arXiv 27 Mar 2024.

## Placed in

- Retrieval-augmented generation / [Privacy and security of RAG](https://mlsanity.com/rag/privacy-and-security-of-rag/)
- Diffusion models / [Robustness, privacy and adversarial aspects](https://mlsanity.com/diffusion/robustness-privacy-and-adversarial-aspects/)
- LLM safety and security / [Watermarking generated content and models](https://mlsanity.com/safety/watermarking-generated-content-and-models/)

## Builds on (most cited first, 30 of 57)

- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [High-Resolution Image Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2112.10752/) (2021)
- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [LoRA: Low-Rank Adaptation of Large Language Models](https://mlsanity.com/paper/2106.09685/) (2021)
- [Denoising Diffusion Implicit Models](https://mlsanity.com/paper/2010.02502/) (2020)
- [Score-Based Generative Modeling through Stochastic Differential Equations](https://mlsanity.com/paper/2011.13456/) (2020)
- [Diffusion Models Beat GANs on Image Synthesis](https://mlsanity.com/paper/2105.05233/) (2021)
- [Hierarchical Text-Conditional Image Generation with CLIP Latents](https://mlsanity.com/paper/2204.06125/) (2022)
- [Classifier-Free Diffusion Guidance](https://mlsanity.com/paper/2207.12598/) (2022)
- [Zero-Shot Text-to-Image Generation](https://mlsanity.com/paper/2102.12092/) (2021)
- [Generative Modeling by Estimating Gradients of the Data Distribution](https://mlsanity.com/paper/1907.05600/) (2019)
- [DreamBooth: Fine Tuning Text-to-Image Diffusion Models for Subject-Driven Generation](https://mlsanity.com/paper/2208.12242/) (2022)
- [InstructPix2Pix: Learning to Follow Image Editing Instructions](https://mlsanity.com/paper/2211.09800/) (2022)
- [Prompt-to-Prompt Image Editing with Cross Attention Control](https://mlsanity.com/paper/2208.01626/) (2022)
- [An Image is Worth One Word: Personalizing Text-to-Image Generation using Textual Inversion](https://mlsanity.com/paper/2208.01618/) (2022)
- [Extracting Training Data from Large Language Models](https://mlsanity.com/paper/2012.07805/) (2020)
- [REALM: Retrieval-Augmented Language Model Pre-Training](https://mlsanity.com/paper/2002.08909/) (2020)
- [Visual Prompt Tuning](https://mlsanity.com/paper/2203.12119/) (2022)
- [Multi-Concept Customization of Text-to-Image Diffusion](https://mlsanity.com/paper/2212.04488/) (2022)
- [Machine Unlearning](https://mlsanity.com/paper/1912.03817/) (2019)
- [eDiff-I: Text-to-Image Diffusion Models with an Ensemble of Expert Denoisers](https://mlsanity.com/paper/2211.01324/) (2022)
- [Generalization through Memorization: Nearest Neighbor Language Models](https://mlsanity.com/paper/1911.00172/) (2019)
- [REPLUG: Retrieval-Augmented Black-Box Language Models](https://mlsanity.com/paper/2301.12652/) (2023)
- [Extracting Training Data from Diffusion Models](https://mlsanity.com/paper/2301.13188/) (2023)
- [In-Context Retrieval-Augmented Language Models](https://mlsanity.com/paper/2302.00083/) (2023)
- [Compositional Visual Generation with Composable Diffusion Models](https://mlsanity.com/paper/2206.01714/) (2022)
- [Eternal Sunshine of the Spotless Net: Selective Forgetting in Deep Networks](https://mlsanity.com/paper/1911.04933/) (2019)
- [Erasing Concepts from Diffusion Models](https://mlsanity.com/paper/2303.07345/) (2023)
- [TIFA: Accurate and Interpretable Text-to-Image Faithfulness Evaluation with Question Answering](https://mlsanity.com/paper/2303.11897/) (2023)
- [Diffusion Art or Digital Forgery? Investigating Data Replication in Diffusion Models](https://mlsanity.com/paper/2212.03860/) (2022)

## Built on by (most cited first, 24 of 24)

- [CopyBench: Measuring Literal and Non-Literal Reproduction of Copyright-Protected Text in Language Model Generation](https://mlsanity.com/paper/2407.07087/) (2024)
- [A Systematic Review of Key Retrieval-Augmented Generation (RAG) Systems: Progress, Gaps, and Future Directions](https://mlsanity.com/paper/2507.18910/) (2025)
- [Fantastic Copyrighted Beasts and How (Not) to Generate Them](https://mlsanity.com/paper/2406.14526/) (2024)
- [Automatic Jailbreaking of the Text-to-Image Generative AI Systems](https://mlsanity.com/paper/2405.16567/) (2024)
- [Signs as Tokens: A Retrieval-Enhanced Multilingual Sign Language Generator](https://mlsanity.com/paper/2411.17799/) (2024)
- [LoTUS: Large-Scale Machine Unlearning with a Taste of Uncertainty](https://mlsanity.com/paper/2503.18314/) (2025)
- [Dataset Protection via Watermarked Canaries in Retrieval-Augmented LLMs](https://mlsanity.com/paper/2502.10673/) (2025)
- [Copyright-Protected Language Generation via Adaptive Model Fusion](https://mlsanity.com/paper/2412.06619/) (2024)
- [Multimodal Unlearning Across Vision, Language, Video, and Audio: Survey of Methods, Datasets, and Benchmarks](https://mlsanity.com/paper/2607.07907/) (2026)
- [The KL3M Data Project: Copyright-Clean Training Resources for Large Language Models](https://mlsanity.com/paper/2504.07854/) (2025)
- [Retrievals Can Be Detrimental: Unveiling the Backdoor Vulnerability of Retrieval-Augmented Diffusion Models](https://mlsanity.com/paper/2501.13340/) (2025)
- [SUV: Scalable Large Language Model Copyright Compliance with Regularized Selective Unlearning](https://mlsanity.com/paper/2503.22948/) (2025)
- [Strong Copyright Protection for Language Models via Adaptive Model Fusion](https://mlsanity.com/paper/2407.20105/) (2024)
- [Negative Token Merging: Image-based Adversarial Feature Guidance](https://mlsanity.com/paper/2412.01339/) (2024)
- [ImageSentinel: Protecting Visual Datasets from Unauthorized Retrieval-Augmented Image Generation](https://mlsanity.com/paper/2510.12119/) (2025)
- [Creating a Taxonomy for Retrieval Augmented Generation Applications](https://mlsanity.com/paper/2408.02854/) (2024)
- [CPSample: Classifier Protected Sampling for Guarding Training Data During Diffusion](https://mlsanity.com/paper/2409.07025/) (2024)
- [External Data Extraction Attacks against Retrieval-Augmented Large Language Models](https://mlsanity.com/paper/2510.02964/) (2025)
- [Guardians of Generation: Dynamic Inference-Time Copyright Shielding with Adaptive Guidance for AI Image Generation](https://mlsanity.com/paper/2503.16171/) (2025)
- [POLYRAG: Integrating Polyviews into Retrieval-Augmented Generation for Medical Applications](https://mlsanity.com/paper/2504.14917/) (2025)
- [Blameless Users in a Clean Room: Defining Copyright Protection for Generative Models](https://mlsanity.com/paper/2506.19881/) (2025)
- [From Architectural Sketch to Conceptual Representation: Using Structure-Aware Diffusion Model to Generate Renderings of School Buildings](https://mlsanity.com/paper/2503.03090/) (2025)
- [SentinelRAG: Synthetic Sentinel Knowledge for RAG Database Copyright Protection](https://mlsanity.com/paper/2606.05787/) (2026)
- [Do More Modalities Always Help? A Geometric Perspective on Missing-Modality Robustness](https://mlsanity.com/paper/2610.04792/) (2026)

## Abstract (arXiv)

Retrieval Augmented Generation (RAG) is emerging as a flexible and robust technique to adapt models to private users data without training, to handle credit attribution, and to allow efficient machine unlearning at scale. However, RAG techniques for image generation may lead to parts of the retrieved samples being copied in the model's output. To reduce risks of leaking private information contained in the retrieved set, we introduce Copy-Protected generation with Retrieval (CPR), a new method for RAG with strong copyright protection guarantees in a mixed-private setting for diffusion models.CPR allows to condition the output of diffusion models on a set of retrieved images, while also guaranteeing that unique identifiable information about those example is not exposed in the generated outputs. In particular, it does so by sampling from a mixture of public (safe) distribution and private (user) distribution by merging their diffusion scores at inference. We prove that CPR satisfies Near Access Freeness (NAF) which bounds the amount of information an attacker may be able to extract from the generated images. We provide two algorithms for copyright protection, CPR-KL and CPR-Choose. Unlike previously proposed rejection-sampling-based NAF methods, our methods enable efficient copyright-protected sampling with a single run of backward diffusion. We show that our method can be applied to any pre-trained conditional diffusion model, such as Stable Diffusion or unCLIP. In particular, we empirically show that applying CPR on top of unCLIP improves quality and text-to-image alignment of the generated results (81.4 to 83.17 on TIFA benchmark), while enabling credit attribution, copy-right protection, and deterministic, constant time, unlearning.