# RAG with Differential Privacy

arXiv: https://arxiv.org/abs/2412.19291 · HTML: https://mlsanity.com/paper/2412.19291/

## Key facts

- Cited by 18 papers in the map (data through 14 Oct 2026).
- Placed in Privacy and security of RAG, one of 15 directions in Retrieval-augmented generation.
- Builds on 9 of its 12 references that are in the map.
- First posted on arXiv 26 Dec 2024.

## Placed in

- Retrieval-augmented generation / [Privacy and security of RAG](https://mlsanity.com/rag/privacy-and-security-of-rag/)

## Builds on (most cited first, 9 of 9)

- [Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks](https://mlsanity.com/paper/2005.11401/) (2020)
- [Deep Learning with Differential Privacy](https://mlsanity.com/paper/1607.00133/) (2016)
- [Membership Inference Attacks against Machine Learning Models](https://mlsanity.com/paper/1610.05820/) (2016)
- [Extracting Training Data from Large Language Models](https://mlsanity.com/paper/2012.07805/) (2020)
- [How to DP-fy ML: A Practical Guide to Machine Learning with Differential Privacy](https://mlsanity.com/paper/2303.00654/) (2023)
- [Follow My Instruction and Spill the Beans: Scalable Data Extraction from Retrieval-Augmented Generation Systems](https://mlsanity.com/paper/2402.17840/) (2024)
- [Differentially Private Synthetic Data via Foundation Model APIs 2: Text](https://mlsanity.com/paper/2403.01749/) (2024)
- [Differentially Private Optimization on Large Model at Small Cost](https://mlsanity.com/paper/2210.00038/) (2022)
- [DP-RDM: Adapting Diffusion Models to Private Domains Without Fine-Tuning](https://mlsanity.com/paper/2403.14421/) (2024)

## Built on by (most cited first, 18 of 18)

- [Silent Leaks: Implicit Knowledge Extraction Attack on RAG Systems through Benign Queries](https://mlsanity.com/paper/2505.15420/) (2025)
- [Privacy-Aware Decoding: Mitigating Privacy Leakage of Large Language Models in Retrieval-Augmented Generation](https://mlsanity.com/paper/2508.03098/) (2025)
- [SoK: Privacy Risks and Mitigations in Retrieval-Augmented Generation Systems](https://mlsanity.com/paper/2601.03979/) (2026)
- [Differentially Private Synthetic Text Generation for Retrieval-Augmented Generation (RAG)](https://mlsanity.com/paper/2510.06719/) (2025)
- [Private-RAG: Answering Multiple Queries with LLMs while Keeping Your Data Private](https://mlsanity.com/paper/2511.07637/) (2025)
- [Privacy Challenges and Solutions in Retrieval-Augmented Generation-Enhanced LLMs for Healthcare Chatbots: A Review of Applications, Risks, and Future Directions](https://mlsanity.com/paper/2511.11347/) (2025)
- [Distributed Retrieval-Augmented Generation](https://mlsanity.com/paper/2505.00443/) (2025)
- [Securing Retrieval-Augmented Generation: A Taxonomy of Attacks, Defenses, and Future Directions](https://mlsanity.com/paper/2604.08304/) (2026)
- [ToolMinimize: Auditing and Rewriting LLM Agent Tool Calls to Minimize Privacy Exposure](https://mlsanity.com/paper/2608.24957/) (2026)
- [A Case Study on the Impact of Anonymization Along the RAG Pipeline](https://mlsanity.com/paper/2604.15958/) (2026)
- [Adaptive Defense Orchestration for RAG: A Sentinel-Strategist Architecture against Multi-Vector Attacks](https://mlsanity.com/paper/2604.20932/) (2026)
- [PRAG: End-to-End Privacy-Preserving Retrieval-Augmented Generation](https://mlsanity.com/paper/2604.26525/) (2026)
- [Barriers to Counterfactual Credit Attribution for Autoregressive Models](https://mlsanity.com/paper/2605.01425/) (2026)
- [Security and Privacy in Retrieval-Augmented Generation: Architectures, Threats, Defenses, and Future Directions for Building Trustworthy Systems](https://mlsanity.com/paper/2606.25533/) (2026)
- [Is External Database Protection Static in Retrieval-Augmented Generation? Rethinking Privacy Preservation under Dynamic Queries](https://mlsanity.com/paper/2607.14811/) (2026)
- [Mitigating Database Leakage in RAG Systems with Keyword-Grounded Fact Substitution](https://mlsanity.com/paper/2608.21656/) (2026)
- [The Privacy-Hallucination Tradeoff in Differentially Private Language Models](https://mlsanity.com/paper/2609.00492/) (2026)
- [Only Pay What You Must Spend: On-Demand Privacy Budget Payment for Differentially Private RAG](https://mlsanity.com/paper/2609.27406/) (2026)

## Abstract (arXiv)

Retrieval-Augmented Generation (RAG) has emerged as the dominant technique to provide Large Language Models (LLM) with fresh and relevant context, mitigating the risk of hallucinations and improving the overall quality of responses in environments with large and fast moving knowledge bases. However, the integration of external documents into the generation process raises significant privacy concerns. Indeed, when added to a prompt, it is not possible to guarantee a response will not inadvertently expose confidential data, leading to potential breaches of privacy and ethical dilemmas. This paper explores a practical solution to this problem suitable to general knowledge extraction from personal data. It shows differentially private token generation is a viable approach to private RAG.