mlsanityData to 14 Oct 2026

Pirates of the RAG: Adaptively Attacking LLMs to Leak Knowledge Bases

arXiv 2412.18295 ↗First posted 24 Dec 2024cs.AIcited by 18

On the map: Retrieval-augmented generation → Privacy and security of RAG

Abstract (arXiv)

The growing ubiquity of Retrieval-Augmented Generation (RAG) systems in several real-world services triggers severe concerns about their security. A RAG system improves the generative capabilities of a Large Language Models (LLM) by a retrieval mechanism which operates on a private knowledge base, whose unintended exposure could lead to severe consequences, including breaches of private and sensitive information. This paper presents a black-box attack to force a RAG system to leak its private knowledge base which, differently from existing approaches, is adaptive and automatic. A relevance-based mechanism and an attacker-side open-source LLM favor the generation of effective queries to leak most of the (hidden) knowledge base. Extensive experimentation proves the quality of the proposed algorithm in different RAG pipelines and domains, comparing to very recent related approaches, which turn out to be either not fully black-box, not adaptive, or not based on open-source models. The findings from our study remark the urgent need for more robust privacy safeguards in the design and deployment of RAG systems.

Key facts. Cited by 18 papers in the map (data through 14 Oct 2026). Placed in Privacy and security of RAG, one of 15 directions in Retrieval-augmented generation. Builds on 28 of its 69 references that are in the map. First posted on arXiv 24 Dec 2024.

Builds on

28 of 69 references.

Closest ancestors · 5

Same direction, newest first

  1. 2024Feedback-Guided Extraction of Knowledge Base from Retrieval-Augmented LLM Applications21 Nov
  2. 2024ConfusedPilot: Confused Deputy Risks in RAG-based LLMs9 Aug
  3. 2024Generating Is Believing: Membership Inference Attacks against Retrieval-Augmented Generation27 Jun
  4. 2024Is My Data in Your Retrieval Database? Membership Inference Attacks Against Retrieval Augmented Generation30 May
  5. 2024Follow My Instruction and Spill the Beans: Scalable Data Extraction from Retrieval-Augmented Generation Systems27 Feb

Other references · 23

  1. 2020Language Models are Few-Shot Learnerscited by 32,419
  2. 2024The Llama 3 Herd of Modelscited by 14,715
  3. 2024GPT-4o System Cardcited by 4,588
  4. 2022Emergent Abilities of Large Language Modelscited by 2,512
  5. 2024Phi-3 Technical Report: A Highly Capable Language Model Locally on Your Phonecited by 1,917
  6. 2022Text Embeddings by Weakly-Supervised Contrastive Pre-trainingcited by 1,253
  7. 2023A Multitask, Multilingual, Multimodal Evaluation of ChatGPT on Reasoning, Hallucination, and Interactivitycited by 894
  8. 2023Towards General Text Embeddings with Multi-stage Contrastive Learningcited by 723
  9. 2022A Survey on In-context Learningcited by 721
  10. 2023Siren's Song in the AI Ocean: A Survey on Hallucination in Large Language Modelscited by 698
  11. 2024A Survey on Large Language Models for Code Generationcited by 658
  12. 2023C-Pack: Packed Resources For General Chinese Embeddingscited by 578
11 more
  1. 2024Retrieval-Augmented Generation for AI-Generated Content: A Surveycited by 354
  2. 2023Mitigating the Alignment Tax of RLHFcited by 183
  3. 2023Multilingual Machine Translation with Large Language Models: Empirical Results and Analysiscited by 160
  4. 2023Evaluating Open-Domain Question Answering in the Era of Large Language Modelscited by 111
  5. 2022Position: Considerations for Differentially Private Learning with Large-Scale Public Pretrainingcited by 88
  6. 2024Trustworthiness in Retrieval-Augmented Generation Systems: A Surveycited by 83
  7. 2024A New Era in LLM Security: Exploring Security Concerns in Real-World LLM-based Systemscited by 77
  8. 2023Towards Better Chain-of-Thought Prompting Strategies: A Surveycited by 62
  9. 2019Adversarial Examples in Modern Machine Learning: A Reviewcited by 36
  10. 2024HealthQ: Unveiling Questioning Capabilities of LLM Chains in Healthcare Conversationscited by 31
  11. 2024Unleashing Worms and Extracting Data: Escalating the Outcome of Attacks against RAG-based Inference in Scale and Severity Using Jailbreakingcited by 30

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 5; 2026: 13.

18 papers.

Sort:

By area:

  1. 2025Unveiling Privacy Risks in LLM Agent Memorycited by 101
  2. 2025Silent Leaks: Implicit Knowledge Extraction Attack on RAG Systems through Benign Queriescited by 19
  3. 2025Privacy-Aware RAG: Secure and Isolated Knowledge Retrievalcited by 10
  4. 2026ADAM: A Systematic Data Extraction Attack on Agent Memory via Adaptive Queryingcited by 5
  5. 2026Observable Channels, Not Just Storage: Evaluating Privacy Leakage in LLM Agent Pipelinescited by 4
  6. 2026InjecMEM: Memory Injection Attack on LLM Agent Memory Systemscited by 4
  7. 2026Query-Efficient Agentic Graph Extraction Attacks on GraphRAG Systemscited by 3
  8. 2026Connect the Dots: Knowledge Graph-Guided Crawler Attack on Retrieval-Augmented Generation Systemscited by 3
  9. 2026MemPot: Defending Against Memory Extraction Attack with Optimized Honeypotscited by 2
  10. 2025LLM in the Middle: A Systematic Review of Threats and Mitigations to Real-World LLM-based Systemscited by 1
  11. 2026The Model's Tell: Measuring Context-Leakage Attack Signals with Behavior Gaugescited by 1
  12. 2025Provably Secure Retrieval-Augmented Generationcited by 0
6 more
  1. 2026Detecting RAG Extraction Attack via Dual-Path Runtime Integrity Gamecited by 0
  2. 2026LeakDojo: Decoding the Leakage Threats of RAG Systemscited by 0
  3. 2026ALDEN: Boosting Private Data Extraction from Retrieval-Augmented Generation Systems via Active Learning and Distribution Estimationcited by 0
  4. 2026Privacy-Preserving RAG via Multi-Agent Semantic Rewriting: Achieving Confidentiality Without Compromising Contextual Fidelitycited by 0
  5. 2026RAG-CT: Mitigating Privacy Risks on Retrieval-Augmented Generation Systems via Scanning Prompt Distributioncited by 0
  6. 2026Walking the Embedding Space: Datastore Extraction from Multimodal RAGcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026Securing Retrieval-Augmented Generation: A Taxonomy of Attacks, Defenses, and Future Directions
  2. 2025RAG Security and Privacy: Formalizing the Threat Model and Attack Surface
  3. 2026Security and Privacy in Retrieval-Augmented Generation: Architectures, Threats, Defenses, and Future Directions for Building Trustworthy Systems
  4. 2025Token-Level Precise Attack on RAG: Searching for the Best Alternatives to Mislead Generation
  5. 2024The Good and The Bad: Exploring Privacy Issues in Retrieval-Augmented Generation (RAG)
  6. 2026Mitigating Database Leakage in RAG Systems with Keyword-Grounded Fact Substitution
  7. 2026SoK: Privacy Risks and Mitigations in Retrieval-Augmented Generation Systems
  8. 2024"Glue pizza and eat rocks" -- Exploiting Vulnerabilities in Retrieval-Augmented Generative Models