mlsanityData to 7 Oct 2026

Silent Leaks: Implicit Knowledge Extraction Attack on RAG Systems through Benign Queries

arXiv 2505.15420 ↗First posted 21 May 2025cs.CR cs.AIcited by 19

On the map: Retrieval-augmented generation → Privacy and security of RAG

also in 1 more
Abstract (arXiv)

Retrieval-Augmented Generation (RAG) systems enhance large language models (LLMs) by incorporating external knowledge bases, but this may expose them to extraction attacks, leading to potential copyright and privacy risks. However, existing extraction methods typically rely on malicious inputs such as prompt injection or jailbreaking, making them easily detectable via input- or output-level detection. In this paper, we introduce Implicit Knowledge Extraction Attack (IKEA), which conducts Knowledge Extraction on RAG systems through benign queries. Specifically, IKEA first leverages anchor concepts-keywords related to internal knowledge-to generate queries with a natural appearance, and then designs two mechanisms that lead anchor concepts to thoroughly "explore" the RAG's knowledge: (1) Experience Reflection Sampling, which samples anchor concepts based on past query-response histories, ensuring their relevance to the topic; (2) Trust Region Directed Mutation, which iteratively mutates anchor concepts under similarity constraints to further exploit the embedding space. Extensive experiments demonstrate IKEA's effectiveness under various defenses, surpassing baselines by over 80% in extraction efficiency and 90% in attack success rate. Moreover, the substitute RAG system built from IKEA's extractions shows comparable performance to the original RAG and outperforms those based on baselines across multiple evaluation tasks, underscoring the stealthy copyright infringement risk in RAG systems.

Key facts. Cited by 19 papers in the map (data through 7 Oct 2026). Placed in Privacy and security of RAG, one of 15 directions in Retrieval-augmented generation. Builds on 37 of its 55 references that are in the map. First posted on arXiv 21 May 2025.

Builds on

37 of 55 references.

Closest ancestors · 9

Same direction, newest first

  1. 2025ControlNET: A Firewall for RAG-based LLM System13 Apr
  2. 2025Riddle Me This! Stealthy Membership Inference for Retrieval-Augmented Generation1 Feb
  3. 2025RAG-WM: An Efficient Black-Box Watermarking Approach for Retrieval-Augmented Generation of Large Language Models9 Jan
  4. 2024RAG with Differential Privacy26 Dec
  5. 2024Pirates of the RAG: Adaptively Attacking LLMs to Leak Knowledge Bases24 Dec
  6. 2024Mitigating the Privacy Issues in Retrieval-Augmented Generation (RAG) via Pure Synthetic Data20 Jun
  7. 2024Is My Data in Your Retrieval Database? Membership Inference Attacks Against Retrieval Augmented Generation30 May
  8. 2024Follow My Instruction and Spill the Beans: Scalable Data Extraction from Retrieval-Augmented Generation Systems27 Feb
  9. 2024The Good and The Bad: Exploring Privacy Issues in Retrieval-Augmented Generation (RAG)23 Feb

Other references · 28

  1. 2023GPT-4 Technical Reportcited by 18,132
  2. 2024The Llama 3 Herd of Modelscited by 14,192
  3. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,160
  4. 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,049
  5. 2024Gemini 1.5: Unlocking multimodal understanding across millions of tokens of contextcited by 3,175
  6. 2023Self-RAG: Learning to Retrieve, Generate, and Critique through Self-Reflectioncited by 1,800
  7. 2020MPNet: Masked and Permuted Pre-training for Language Understandingcited by 973
  8. 2023In-Context Retrieval-Augmented Language Modelscited by 694
  9. 2023Active Retrieval Augmented Generationcited by 506
  10. 2023Detecting Language Model Attacks with Perplexitycited by 411
  11. 2024Retrieval-Augmented Generation for AI-Generated Content: A Surveycited by 352
  12. 2023Enhancing Retrieval-Augmented Large Language Models with Iterative Retrieval-Generation Synergycited by 343
16 more
  1. 2023LLM Self Defense: By Self Examination, LLMs Know They Are Being Trickedcited by 226
  2. 2024AutoDefense: Multi-Agent LLM Defense against Jailbreak Attackscited by 135
  3. 2022Re2G: Retrieve, Rerank, Generatecited by 102
  4. 2023R2Former: Unified Retrieval and Reranking Transformer for Place Recognitioncited by 102
  5. 2024MMed-RAG: Versatile Multimodal RAG System for Medical Vision Language Modelscited by 84
  6. 2024CBR-RAG: Case-Based Reasoning for Retrieval Augmented Generation in LLMs for Legal Question Answeringcited by 70
  7. 2024Bridging the Preference Gap between Retrievers and LLMscited by 69
  8. 2021Rethink Training of BERT Rerankers in Multi-Stage Retrieval Pipelinecited by 65
  9. 2024Intention Analysis Makes LLMs A Good Jailbreak Defendercited by 64
  10. 2024Improving Retrieval for RAG based Question Answering Models on Financial Documentscited by 48
  11. 2024AlphaFin: Benchmarking Financial Analysis with Retrieval-Augmented Stock-Chain Frameworkcited by 47
  12. 2024Unleashing Worms and Extracting Data: Escalating the Outcome of Attacks against RAG-based Inference in Scale and Severity Using Jailbreakingcited by 30
  13. 2023Clinfo.ai: An Open-Source Retrieval-Augmented Large Language Model System for Answering Medical Questions using Scientific Literaturecited by 29
  14. 2024REALM: RAG-Driven Enhancement of Multimodal Electronic Health Records Analysis via Large Language Modelscited by 21
  15. 2025EnronQA: Towards Personalized RAG over Private Documentscited by 8
  16. 2024Retail-GPT: leveraging Retrieval Augmented Generation (RAG) for building E-commerce Chat Assistantscited by 2

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 3; 2026: 16.

19 papers.

Sort:

By area:

  1. 2025A Comprehensive Survey on Trustworthiness in Reasoning with Large Language Modelscited by 22
  2. 2025Differentially Private Synthetic Text Generation for Retrieval-Augmented Generation (RAG)cited by 5
  3. 2026Five Queries Are Enough: Query-Efficient and Surrogate-Free Membership Inference Attacks on RAG via Entailmentcited by 4
  4. 2026Query-Efficient Agentic Graph Extraction Attacks on GraphRAG Systemscited by 3
  5. 2026Connect the Dots: Knowledge Graph-Guided Crawler Attack on Retrieval-Augmented Generation Systemscited by 3
  6. 2026InjecMEM: Memory Injection Attack on LLM Agent Memory Systemscited by 3
  7. 2025Privacy Challenges and Solutions in Retrieval-Augmented Generation-Enhanced LLMs for Healthcare Chatbots: A Review of Applications, Risks, and Future Directionscited by 2
  8. 2026Making Theft Useless: Adulteration-Based Protection of Proprietary Knowledge Graphs in GraphRAG Systemscited by 2
  9. 2026Securing Retrieval-Augmented Generation: A Taxonomy of Attacks, Defenses, and Future Directionscited by 2
  10. 2026Isolated but Exposed: Persistence-Based Memory Extraction Attack on LLM Agentscited by 2
  11. 2026Benchmarking Knowledge-Extraction Attack and Defense on Retrieval-Augmented Generationcited by 1
  12. 2026Hierarchical Alignment: Enforcing Hierarchical Instruction-Following in LLMs through Logical Consistencycited by 1
7 more
  1. 2026LeakDojo: Decoding the Leakage Threats of RAG Systemscited by 0
  2. 2026ALDEN: Boosting Private Data Extraction from Retrieval-Augmented Generation Systems via Active Learning and Distribution Estimationcited by 0
  3. 2026AliMark: Enhancing Robustness of Sentence-Level Watermarking Against Text Paraphrasingcited by 0
  4. 2026Extracting Knowledge from Tools in LLM Agentscited by 0
  5. 2026Membership Inference in Fine-tuned Diffusion Language Models via Token-level Memorization Asymmetrycited by 0
  6. 2026Context Inference Attacks Without Jailbreakscited by 0
  7. 2026Mapping the RAG Landscape: A Four Axis Taxonomy of Efficiency, Defense, Interactivity, and Reasoningcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024PoisonedRAG: Knowledge Corruption Attacks to Retrieval-Augmented Generation of Large Language Models
  2. 2024Data Extraction Attacks in Retrieval-Augmented Generation via Backdoors
  3. 2026Mitigating Database Leakage in RAG Systems with Keyword-Grounded Fact Substitution
  4. 2025One Shot Dominance: Knowledge Poisoning Attack on Retrieval-Augmented Generation Systems
  5. 2025Rescuing the Unpoisoned: Efficient Defense against Knowledge Corruption Attacks on RAG Systems
  6. 2025Token-Level Precise Attack on RAG: Searching for the Best Alternatives to Mislead Generation
  7. 2026Security and Privacy in Retrieval-Augmented Generation: Architectures, Threats, Defenses, and Future Directions for Building Trustworthy Systems
  8. 2024"Glue pizza and eat rocks" -- Exploiting Vulnerabilities in Retrieval-Augmented Generative Models