mlsanityData to 7 Oct 2026

The Good and The Bad: Exploring Privacy Issues in Retrieval-Augmented Generation (RAG)

arXiv 2402.16893 ↗First posted 23 Feb 2024cs.CR cs.AI cs.CLcited by 159

On the map: Retrieval-augmented generation → Privacy and security of RAG

Abstract (arXiv)

Retrieval-augmented generation (RAG) is a powerful technique to facilitate language model with proprietary and private data, where data privacy is a pivotal concern. Whereas extensive research has demonstrated the privacy risks of large language models (LLMs), the RAG technique could potentially reshape the inherent behaviors of LLM generation, posing new privacy issues that are currently under-explored. In this work, we conduct extensive empirical studies with novel attack methods, which demonstrate the vulnerability of RAG systems on leaking the private retrieval database. Despite the new risk brought by RAG on the retrieval data, we further reveal that RAG can mitigate the leakage of the LLMs' training data. Overall, we provide new insights in this paper for privacy protection of retrieval-augmented LLMs, which benefit both LLMs and RAG systems builders. Our code is available at https://github.com/phycholosogy/RAG-privacy.

Key facts. Cited by 159 papers in the map (data through 7 Oct 2026). Placed in Privacy and security of RAG, one of 15 directions in Retrieval-augmented generation. Builds on 22 of its 33 references that are in the map. First posted on arXiv 23 Feb 2024.

Builds on

22 of 33 references.

References · 22

  1. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,160
  2. 2023Retrieval-Augmented Generation for Large Language Models: A Surveycited by 2,185
  3. 2020Extracting Training Data from Large Language Modelscited by 2,144
  4. 2021Improving language models by retrieving from trillions of tokenscited by 1,278
  5. 2021An Explanation of In-context Learning as Implicit Bayesian Inferencecited by 915
  6. 2019Generalization through Memorization: Nearest Neighbor Language Modelscited by 788
  7. 2023REPLUG: Retrieval-Augmented Black-Box Language Modelscited by 786
  8. 2021Deduplicating Training Data Makes Language Models Bettercited by 705
  9. 2023In-Context Retrieval-Augmented Language Modelscited by 694
  10. 2021Retrieval Augmentation Reduces Hallucination in Conversationcited by 691
  11. 2023Enhancing Retrieval-Augmented Large Language Models with Iterative Retrieval-Generation Synergycited by 343
  12. 2022Deduplicating Training Data Mitigates Privacy Risks in Language Modelscited by 309
10 more
  1. 2021Counterfactual Memorization in Neural Language Modelscited by 155
  2. 2021Retrieval Augmented Code Generation and Summarizationcited by 148
  3. 2019Privacy- and Utility-Preserving Textual Analysis via Calibrated Multivariate Perturbationscited by 115
  4. 2023Lift Yourself Up: Retrieval-augmented Text Generation with Self Memorycited by 103
  5. 2023Locally Differentially Private Document Generation Using Zero Shot Promptingcited by 63
  6. 2023Exploring Memorization in Fine-tuned Language Modelscited by 44
  7. 2021TEM: High Utility Metric Differential Privacy on Textcited by 43
  8. 2022Memorization in NLP Fine-tuning Methodscited by 43
  9. 2024Copyright Protection in Generative AI: A Technical Perspectivecited by 35
  10. 2024Reinforcement Learning for Optimizing RAG for Domain Chatbotscited by 26

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 31; 2025: 63; 2026: 65.

159 papers.

Sort:

By area:

  1. 2024A Survey on RAG Meeting LLMs: Towards Retrieval-Augmented Large Language Modelscited by 450
  2. 2024Retrieval-Augmented Generation with Graphs (GraphRAG)cited by 170
  3. 2024AI Agents Under Threat: A Survey of Key Security Challenges and Future Pathwayscited by 122
  4. 2025A Comprehensive Survey in LLM(-Agent) Full Stack Safety: Data, Training and Deploymentcited by 110
  5. 2024InstructRAG: Instructing Retrieval-Augmented Generation via Self-Synthesized Rationalescited by 100
  6. 2025Unveiling Privacy Risks in LLM Agent Memorycited by 99
  7. 2025A Survey on Trustworthy LLM Agents: Threats and Countermeasurescited by 93
  8. 2024BadRAG: Identifying Vulnerabilities in Retrieval Augmented Generation of Large Language Modelscited by 84
  9. 2024Trustworthiness in Retrieval-Augmented Generation Systems: A Surveycited by 83
  10. 2024Follow My Instruction and Spill the Beans: Scalable Data Extraction from Retrieval-Augmented Generation Systemscited by 76
  11. 2025From AI for Science to Agentic Science: A Survey on Autonomous Scientific Discoverycited by 74
  12. 2024Navigating the Risks: A Survey of Security, Privacy, and Ethics Threats in LLM-Based Agentscited by 63
38 more
  1. 2024Mitigating the Privacy Issues in Retrieval-Augmented Generation (RAG) via Pure Synthetic Datacited by 52
  2. 2025Commercial LLM Agents Are Already Vulnerable to Simple Yet Dangerous Attackscited by 49
  3. 2024Is My Data in Your Retrieval Database? Membership Inference Attacks Against Retrieval Augmented Generationcited by 44
  4. 2024Privacy-Preserving Retrieval-Augmented Generation with Differential Privacycited by 38
  5. 2024Feedback-Guided Extraction of Knowledge Base from Retrieval-Augmented LLM Applicationscited by 36
  6. 2024Unleashing Worms and Extracting Data: Escalating the Outcome of Attacks against RAG-based Inference in Scale and Severity Using Jailbreakingcited by 30
  7. 2024Undesirable Memorization in Large Language Models: A Surveycited by 29
  8. 2024Mask-based Membership Inference Attacks for Retrieval-Augmented Generationcited by 29
  9. 2025Towards Trustworthy Retrieval Augmented Generation for Large Language Models: A Surveycited by 29
  10. 2025We Should Identify and Mitigate Third-Party Safety Risks in MCP-Powered Agent Systemscited by 27
  11. 2025Riddle Me This! Stealthy Membership Inference for Retrieval-Augmented Generationcited by 26
  12. 2025Retrieval-Augmented Generation: A Comprehensive Survey of Architectures, Enhancements, and Robustness Frontierscited by 26
  13. 2024Generating Is Believing: Membership Inference Attacks against Retrieval-Augmented Generationcited by 25
  14. 2025FlippedRAG: Black-Box Opinion Manipulation Adversarial Attacks to Retrieval-Augmented Generation Modelscited by 24
  15. 2024Prompt Leakage effect and defense strategies for multi-turn LLM interactionscited by 23
  16. 2024Data Extraction Attacks in Retrieval-Augmented Generation via Backdoorscited by 23
  17. 2025RAG LLMs are Not Safer: A Safety Analysis of Retrieval-Augmented Generation for Large Language Modelscited by 23
  18. 2024ConfusedPilot: Confused Deputy Risks in RAG-based LLMscited by 22
  19. 2024RAG-RewardBench: Benchmarking Reward Models in Retrieval Augmented Generation for Preference Alignmentcited by 22
  20. 2024Towards Knowledge Checking in Retrieval-augmented Generation: A Representation Perspectivecited by 20
  21. 2025Retrieval Augmented Generation Evaluation in the Era of Large Language Models: A Comprehensive Surveycited by 20
  22. 2025Silent Leaks: Implicit Knowledge Extraction Attack on RAG Systems through Benign Queriescited by 19
  23. 2024RemoteRAG: A Privacy-Preserving LLM Cloud RAG Servicecited by 17
  24. 2024Ward: Provable RAG Dataset Inference via LLM Watermarkscited by 15
  25. 2025Empowering GraphRAG with Knowledge Filtering and Integrationcited by 15
  26. 2025A False Sense of Privacy: Evaluating Textual Data Sanitization Beyond Surface-level Privacy Leakagecited by 15
  27. 2026Security Threat Modeling for Emerging AI-Agent Protocols: A Comparative Analysis of MCP, A2A, Agora, and ANPcited by 14
  28. 2025Poisoned-MRAG: Knowledge Poisoning Attacks to Multimodal Retrieval Augmented Generationcited by 12
  29. 2026When Lower Privileges Suffice: Investigating Over-Privileged Tool Selection in LLM Agentscited by 12
  30. 2024LLaVA Needs More Knowledge: Retrieval Augmented Natural Language Generation with Knowledge Graph for Explaining Thoracic Pathologiescited by 11
  31. 2025Privacy-Aware RAG: Secure and Isolated Knowledge Retrievalcited by 10
  32. 2025PIG: Privacy Jailbreak Attack on LLMs via Gradient-based Iterative In-Context Optimizationcited by 10
  33. 2025Tricking Retrievers with Influential Tokens: An Efficient Black-Box Corpus Poisoning Attackcited by 9
  34. 2024Blockchain for Large Language Model Security and Safety: A Holistic Surveycited by 8
  35. 2024Risks, Causes, and Mitigations of Widespread Deployments of Large Language Models (LLMs): A Surveycited by 8
  36. 2025Safeguarding Privacy of Retrieval Data against Membership Inference Attacks: Is This Query Too Close to Home?cited by 8
  37. 2025Privacy-Aware Decoding: Mitigating Privacy Leakage of Large Language Models in Retrieval-Augmented Generationcited by 8
  38. 2025RAG Security and Privacy: Formalizing the Threat Model and Attack Surfacecited by 8

50 most cited of 159 All 159 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2024RAG with Differential Privacy
  2. 2026Towards Secure Retrieval-Augmented Generation: A Comprehensive Review of Threats, Defenses and Benchmarks
  3. 2024Pirates of the RAG: Adaptively Attacking LLMs to Leak Knowledge Bases
  4. 2026Differentially Private Retrieval-Augmented Generation
  5. 2026SD-RAG: A Prompt-Injection-Resilient Framework for Selective Disclosure in Retrieval-Augmented Generation
  6. 2026P2RAG: Efficient Privacy-Preserving RAG Service Supporting Arbitrary Top-k Retrieval
  7. 2024No Free Lunch: Retrieval-Augmented Generation Undermines Fairness in LLMs, Even for Vigilant Users
  8. 2025Secure Multifaceted-RAG for Enterprise: Hybrid Knowledge Retrieval with Security Filtering