mlsanityData to 14 Oct 2026

LeakDojo: Decoding the Leakage Threats of RAG Systems

arXiv 2605.05818 ↗First posted 7 May 2026cs.CR cs.AI cs.CLcited by 0

On the map: Retrieval-augmented generation → Privacy and security of RAG

Abstract (arXiv)

Retrieval-Augmented Generation (RAG) enables large language models (LLMs) to leverage external knowledge, but also exposes valuable RAG databases to leakage attacks. As RAG systems grow more complex and LLMs exhibit stronger instruction-following capabilities, existing studies fall short of systematically assessing RAG leakage risks. We present LeakDojo, a configurable framework for controlled evaluation of RAG leakage. Using LeakDojo, we benchmark six existing attacks across fourteen LLMs, four datasets, and diverse RAG systems. Our study reveals that (1) query generation and adversarial instructions contribute independently to leakage, with overall leakage well approximated by their product; (2) stronger instruction-following capability correlates with higher leakage risk; and (3) improvements in RAG faithfulness can introduce increased leakage risk. These findings provide actionable insights for understanding and mitigating RAG leakage in practice. Our codebase is available at https://github.com/yeasen-z/LeakDojo.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Privacy and security of RAG, one of 15 directions in Retrieval-augmented generation. Builds on 24 of its 42 references that are in the map. First posted on arXiv 7 May 2026.

Builds on

24 of 42 references.

Closest ancestors · 5

Same direction, newest first

  1. 2025Silent Leaks: Implicit Knowledge Extraction Attack on RAG Systems through Benign Queries21 May
  2. 2024Pirates of the RAG: Adaptively Attacking LLMs to Leak Knowledge Bases24 Dec
  3. 2024Feedback-Guided Extraction of Knowledge Base from Retrieval-Augmented LLM Applications21 Nov
  4. 2024Follow My Instruction and Spill the Beans: Scalable Data Extraction from Retrieval-Augmented Generation Systems27 Feb
  5. 2024The Good and The Bad: Exploring Privacy Issues in Retrieval-Augmented Generation (RAG)23 Feb

Other references · 19

  1. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,297
  2. 2020REALM: Retrieval-Augmented Language Model Pre-Trainingcited by 2,141
  3. 2021BEIR: A Heterogenous Benchmark for Zero-shot Evaluation of Information Retrieval Modelscited by 1,252
  4. 2020ColBERT: Efficient and Effective Passage Search via Contextualized Late Interaction over BERTcited by 1,178
  5. 2022Ignore Previous Prompt: Attack Techniques For Language Modelscited by 832
  6. 2020Fact or Fiction: Verifying Scientific Claimscited by 554
  7. 2024Does Fine-Tuning LLMs on New Knowledge Encourage Hallucinations?cited by 232
  8. 2023Tensor Trust: Interpretable Prompt Injection Attacks from an Online Gamecited by 145
  9. 2023Effective Prompt Extraction from Language Modelscited by 85
  10. 2024MMed-RAG: Versatile Multimodal RAG System for Medical Vision Language Modelscited by 85
  11. 2024CBR-RAG: Case-Based Reasoning for Retrieval Augmented Generation in LLMs for Legal Question Answeringcited by 72
  12. 2023RewriteLM: An Instruction-Tuned Large Language Model for Text Rewritingcited by 67
7 more
  1. 2024Improving Retrieval for RAG based Question Answering Models on Financial Documentscited by 49
  2. 2024RaFe: Ranking Feedback Improves Query Rewriting for RAGcited by 49
  3. 2024Modular RAG: Transforming RAG Systems into LEGO-like Reconfigurable Frameworkscited by 42
  4. 2024Unleashing Worms and Extracting Data: Escalating the Outcome of Attacks against RAG-based Inference in Scale and Severity Using Jailbreakingcited by 30
  5. 2024Context Embeddings for Efficient Answer Generation in RAGcited by 26
  6. 2024Raccoon: Prompt Extraction Benchmark of LLM-Integrated Applicationscited by 16
  7. 2024Refiner: Restructure Retrieval Content Efficiently to Advance Question-Answering Capabilitiescited by 16

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025RAG Security and Privacy: Formalizing the Threat Model and Attack Surface
  2. 2026Retrieved But Not Reliable: A Survey on Attacks, and Defenses in Retrieval-Augmented Generation
  3. 2026Mitigating Database Leakage in RAG Systems with Keyword-Grounded Fact Substitution
  4. 2026Towards Secure Retrieval-Augmented Generation: A Comprehensive Review of Threats, Defenses and Benchmarks
  5. 2025Token-Level Precise Attack on RAG: Searching for the Best Alternatives to Mislead Generation
  6. 2025BudgetLeak: Membership Inference Attacks on RAG Systems via the Generation Budget Side Channel
  7. 2026SoK: Privacy Risks and Mitigations in Retrieval-Augmented Generation Systems
  8. 2026Security and Privacy in Retrieval-Augmented Generation: Architectures, Threats, Defenses, and Future Directions for Building Trustworthy Systems