# The Good and The Bad: Exploring Privacy Issues in Retrieval-Augmented Generation (RAG)

arXiv: https://arxiv.org/abs/2402.16893 · HTML: https://mlsanity.com/paper/2402.16893/

## Key facts

- Cited by 159 papers in the map (data through 14 Oct 2026).
- Placed in Privacy and security of RAG, one of 15 directions in Retrieval-augmented generation.
- Builds on 22 of its 33 references that are in the map.
- First posted on arXiv 23 Feb 2024.

## Placed in

- Retrieval-augmented generation / [Privacy and security of RAG](https://mlsanity.com/rag/privacy-and-security-of-rag/)

## Builds on (most cited first, 22 of 22)

- [Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks](https://mlsanity.com/paper/2005.11401/) (2020)
- [Retrieval-Augmented Generation for Large Language Models: A Survey](https://mlsanity.com/paper/2312.10997/) (2023)
- [Extracting Training Data from Large Language Models](https://mlsanity.com/paper/2012.07805/) (2020)
- [Improving language models by retrieving from trillions of tokens](https://mlsanity.com/paper/2112.04426/) (2021)
- [An Explanation of In-context Learning as Implicit Bayesian Inference](https://mlsanity.com/paper/2111.02080/) (2021)
- [Generalization through Memorization: Nearest Neighbor Language Models](https://mlsanity.com/paper/1911.00172/) (2019)
- [REPLUG: Retrieval-Augmented Black-Box Language Models](https://mlsanity.com/paper/2301.12652/) (2023)
- [Deduplicating Training Data Makes Language Models Better](https://mlsanity.com/paper/2107.06499/) (2021)
- [In-Context Retrieval-Augmented Language Models](https://mlsanity.com/paper/2302.00083/) (2023)
- [Retrieval Augmentation Reduces Hallucination in Conversation](https://mlsanity.com/paper/2104.07567/) (2021)
- [Enhancing Retrieval-Augmented Large Language Models with Iterative Retrieval-Generation Synergy](https://mlsanity.com/paper/2305.15294/) (2023)
- [Deduplicating Training Data Mitigates Privacy Risks in Language Models](https://mlsanity.com/paper/2202.06539/) (2022)
- [Counterfactual Memorization in Neural Language Models](https://mlsanity.com/paper/2112.12938/) (2021)
- [Retrieval Augmented Code Generation and Summarization](https://mlsanity.com/paper/2108.11601/) (2021)
- [Privacy- and Utility-Preserving Textual Analysis via Calibrated Multivariate Perturbations](https://mlsanity.com/paper/1910.08902/) (2019)
- [Lift Yourself Up: Retrieval-augmented Text Generation with Self Memory](https://mlsanity.com/paper/2305.02437/) (2023)
- [Locally Differentially Private Document Generation Using Zero Shot Prompting](https://mlsanity.com/paper/2310.16111/) (2023)
- [Memorization in NLP Fine-tuning Methods](https://mlsanity.com/paper/2205.12506/) (2022)
- [Exploring Memorization in Fine-tuned Language Models](https://mlsanity.com/paper/2310.06714/) (2023)
- [TEM: High Utility Metric Differential Privacy on Text](https://mlsanity.com/paper/2107.07928/) (2021)
- [Copyright Protection in Generative AI: A Technical Perspective](https://mlsanity.com/paper/2402.02333/) (2024)
- [Reinforcement Learning for Optimizing RAG for Domain Chatbots](https://mlsanity.com/paper/2401.06800/) (2024)

## Built on by (most cited first, 30 of 159)

- [A Survey on RAG Meeting LLMs: Towards Retrieval-Augmented Large Language Models](https://mlsanity.com/paper/2405.06211/) (2024)
- [Retrieval-Augmented Generation with Graphs (GraphRAG)](https://mlsanity.com/paper/2501.00309/) (2024)
- [AI Agents Under Threat: A Survey of Key Security Challenges and Future Pathways](https://mlsanity.com/paper/2406.02630/) (2024)
- [A Comprehensive Survey in LLM(-Agent) Full Stack Safety: Data, Training and Deployment](https://mlsanity.com/paper/2504.15585/) (2025)
- [InstructRAG: Instructing Retrieval-Augmented Generation via Self-Synthesized Rationales](https://mlsanity.com/paper/2406.13629/) (2024)
- [Unveiling Privacy Risks in LLM Agent Memory](https://mlsanity.com/paper/2502.13172/) (2025)
- [A Survey on Trustworthy LLM Agents: Threats and Countermeasures](https://mlsanity.com/paper/2503.09648/) (2025)
- [BadRAG: Identifying Vulnerabilities in Retrieval Augmented Generation of Large Language Models](https://mlsanity.com/paper/2406.00083/) (2024)
- [Trustworthiness in Retrieval-Augmented Generation Systems: A Survey](https://mlsanity.com/paper/2409.10102/) (2024)
- [From AI for Science to Agentic Science: A Survey on Autonomous Scientific Discovery](https://mlsanity.com/paper/2508.14111/) (2025)
- [Follow My Instruction and Spill the Beans: Scalable Data Extraction from Retrieval-Augmented Generation Systems](https://mlsanity.com/paper/2402.17840/) (2024)
- [Navigating the Risks: A Survey of Security, Privacy, and Ethics Threats in LLM-Based Agents](https://mlsanity.com/paper/2411.09523/) (2024)
- [Mitigating the Privacy Issues in Retrieval-Augmented Generation (RAG) via Pure Synthetic Data](https://mlsanity.com/paper/2406.14773/) (2024)
- [Commercial LLM Agents Are Already Vulnerable to Simple Yet Dangerous Attacks](https://mlsanity.com/paper/2502.08586/) (2025)
- [Is My Data in Your Retrieval Database? Membership Inference Attacks Against Retrieval Augmented Generation](https://mlsanity.com/paper/2405.20446/) (2024)
- [Privacy-Preserving Retrieval-Augmented Generation with Differential Privacy](https://mlsanity.com/paper/2412.04697/) (2024)
- [Feedback-Guided Extraction of Knowledge Base from Retrieval-Augmented LLM Applications](https://mlsanity.com/paper/2411.14110/) (2024)
- [Unleashing Worms and Extracting Data: Escalating the Outcome of Attacks against RAG-based Inference in Scale and Severity Using Jailbreaking](https://mlsanity.com/paper/2409.08045/) (2024)
- [Towards Trustworthy Retrieval Augmented Generation for Large Language Models: A Survey](https://mlsanity.com/paper/2502.06872/) (2025)
- [Undesirable Memorization in Large Language Models: A Survey](https://mlsanity.com/paper/2410.02650/) (2024)
- [Mask-based Membership Inference Attacks for Retrieval-Augmented Generation](https://mlsanity.com/paper/2410.20142/) (2024)
- [Retrieval-Augmented Generation: A Comprehensive Survey of Architectures, Enhancements, and Robustness Frontiers](https://mlsanity.com/paper/2506.00054/) (2025)
- [We Should Identify and Mitigate Third-Party Safety Risks in MCP-Powered Agent Systems](https://mlsanity.com/paper/2506.13666/) (2025)
- [Riddle Me This! Stealthy Membership Inference for Retrieval-Augmented Generation](https://mlsanity.com/paper/2502.00306/) (2025)
- [Generating Is Believing: Membership Inference Attacks against Retrieval-Augmented Generation](https://mlsanity.com/paper/2406.19234/) (2024)
- [FlippedRAG: Black-Box Opinion Manipulation Adversarial Attacks to Retrieval-Augmented Generation Models](https://mlsanity.com/paper/2501.02968/) (2025)
- [RAG LLMs are Not Safer: A Safety Analysis of Retrieval-Augmented Generation for Large Language Models](https://mlsanity.com/paper/2504.18041/) (2025)
- [Prompt Leakage effect and defense strategies for multi-turn LLM interactions](https://mlsanity.com/paper/2404.16251/) (2024)
- [Data Extraction Attacks in Retrieval-Augmented Generation via Backdoors](https://mlsanity.com/paper/2411.01705/) (2024)
- [RAG-RewardBench: Benchmarking Reward Models in Retrieval Augmented Generation for Preference Alignment](https://mlsanity.com/paper/2412.13746/) (2024)

## Abstract (arXiv)

Retrieval-augmented generation (RAG) is a powerful technique to facilitate language model with proprietary and private data, where data privacy is a pivotal concern. Whereas extensive research has demonstrated the privacy risks of large language models (LLMs), the RAG technique could potentially reshape the inherent behaviors of LLM generation, posing new privacy issues that are currently under-explored. In this work, we conduct extensive empirical studies with novel attack methods, which demonstrate the vulnerability of RAG systems on leaking the private retrieval database. Despite the new risk brought by RAG on the retrieval data, we further reveal that RAG can mitigate the leakage of the LLMs' training data. Overall, we provide new insights in this paper for privacy protection of retrieval-augmented LLMs, which benefit both LLMs and RAG systems builders. Our code is available at https://github.com/phycholosogy/RAG-privacy.