# Silent Leaks: Implicit Knowledge Extraction Attack on RAG Systems through Benign Queries

arXiv: https://arxiv.org/abs/2505.15420 · HTML: https://mlsanity.com/paper/2505.15420/

## Key facts

- Cited by 19 papers in the map (data through 14 Oct 2026).
- Placed in Privacy and security of RAG, one of 15 directions in Retrieval-augmented generation.
- Builds on 37 of its 55 references that are in the map.
- First posted on arXiv 21 May 2025.

## Placed in

- Retrieval-augmented generation / [Privacy and security of RAG](https://mlsanity.com/rag/privacy-and-security-of-rag/)
- LLM safety and security / [Membership inference attacks](https://mlsanity.com/safety/membership-inference-attacks/)

## Builds on (most cited first, 30 of 37)

- [GPT-4 Technical Report](https://mlsanity.com/paper/2303.08774/) (2023)
- [The Llama 3 Herd of Models](https://mlsanity.com/paper/2407.21783/) (2024)
- [Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks](https://mlsanity.com/paper/2005.11401/) (2020)
- [Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks](https://mlsanity.com/paper/1908.10084/) (2019)
- [Gemini 1.5: Unlocking multimodal understanding across millions of tokens of context](https://mlsanity.com/paper/2403.05530/) (2024)
- [Self-RAG: Learning to Retrieve, Generate, and Critique through Self-Reflection](https://mlsanity.com/paper/2310.11511/) (2023)
- [MPNet: Masked and Permuted Pre-training for Language Understanding](https://mlsanity.com/paper/2004.09297/) (2020)
- [In-Context Retrieval-Augmented Language Models](https://mlsanity.com/paper/2302.00083/) (2023)
- [Active Retrieval Augmented Generation](https://mlsanity.com/paper/2305.06983/) (2023)
- [Detecting Language Model Attacks with Perplexity](https://mlsanity.com/paper/2308.14132/) (2023)
- [Retrieval-Augmented Generation for AI-Generated Content: A Survey](https://mlsanity.com/paper/2402.19473/) (2024)
- [Enhancing Retrieval-Augmented Large Language Models with Iterative Retrieval-Generation Synergy](https://mlsanity.com/paper/2305.15294/) (2023)
- [LLM Self Defense: By Self Examination, LLMs Know They Are Being Tricked](https://mlsanity.com/paper/2308.07308/) (2023)
- [The Good and The Bad: Exploring Privacy Issues in Retrieval-Augmented Generation (RAG)](https://mlsanity.com/paper/2402.16893/) (2024)
- [AutoDefense: Multi-Agent LLM Defense against Jailbreak Attacks](https://mlsanity.com/paper/2403.04783/) (2024)
- [Re2G: Retrieve, Rerank, Generate](https://mlsanity.com/paper/2207.06300/) (2022)
- [R2Former: Unified Retrieval and Reranking Transformer for Place Recognition](https://mlsanity.com/paper/2304.03410/) (2023)
- [MMed-RAG: Versatile Multimodal RAG System for Medical Vision Language Models](https://mlsanity.com/paper/2410.13085/) (2024)
- [Follow My Instruction and Spill the Beans: Scalable Data Extraction from Retrieval-Augmented Generation Systems](https://mlsanity.com/paper/2402.17840/) (2024)
- [Intention Analysis Makes LLMs A Good Jailbreak Defender](https://mlsanity.com/paper/2401.06561/) (2024)
- [CBR-RAG: Case-Based Reasoning for Retrieval Augmented Generation in LLMs for Legal Question Answering](https://mlsanity.com/paper/2404.04302/) (2024)
- [Bridging the Preference Gap between Retrievers and LLMs](https://mlsanity.com/paper/2401.06954/) (2024)
- [Rethink Training of BERT Rerankers in Multi-Stage Retrieval Pipeline](https://mlsanity.com/paper/2101.08751/) (2021)
- [Mitigating the Privacy Issues in Retrieval-Augmented Generation (RAG) via Pure Synthetic Data](https://mlsanity.com/paper/2406.14773/) (2024)
- [Improving Retrieval for RAG based Question Answering Models on Financial Documents](https://mlsanity.com/paper/2404.07221/) (2024)
- [AlphaFin: Benchmarking Financial Analysis with Retrieval-Augmented Stock-Chain Framework](https://mlsanity.com/paper/2403.12582/) (2024)
- [Is My Data in Your Retrieval Database? Membership Inference Attacks Against Retrieval Augmented Generation](https://mlsanity.com/paper/2405.20446/) (2024)
- [Unleashing Worms and Extracting Data: Escalating the Outcome of Attacks against RAG-based Inference in Scale and Severity Using Jailbreaking](https://mlsanity.com/paper/2409.08045/) (2024)
- [Clinfo.ai: An Open-Source Retrieval-Augmented Large Language Model System for Answering Medical Questions using Scientific Literature](https://mlsanity.com/paper/2310.16146/) (2023)
- [Riddle Me This! Stealthy Membership Inference for Retrieval-Augmented Generation](https://mlsanity.com/paper/2502.00306/) (2025)

## Built on by (most cited first, 19 of 19)

- [A Comprehensive Survey on Trustworthiness in Reasoning with Large Language Models](https://mlsanity.com/paper/2509.03871/) (2025)
- [Differentially Private Synthetic Text Generation for Retrieval-Augmented Generation (RAG)](https://mlsanity.com/paper/2510.06719/) (2025)
- [Five Queries Are Enough: Query-Efficient and Surrogate-Free Membership Inference Attacks on RAG via Entailment](https://mlsanity.com/paper/2605.24312/) (2026)
- [InjecMEM: Memory Injection Attack on LLM Agent Memory Systems](https://mlsanity.com/paper/2608.23471/) (2026)
- [Privacy Challenges and Solutions in Retrieval-Augmented Generation-Enhanced LLMs for Healthcare Chatbots: A Review of Applications, Risks, and Future Directions](https://mlsanity.com/paper/2511.11347/) (2025)
- [Query-Efficient Agentic Graph Extraction Attacks on GraphRAG Systems](https://mlsanity.com/paper/2601.14662/) (2026)
- [Connect the Dots: Knowledge Graph-Guided Crawler Attack on Retrieval-Augmented Generation Systems](https://mlsanity.com/paper/2601.15678/) (2026)
- [Making Theft Useless: Adulteration-Based Protection of Proprietary Knowledge Graphs in GraphRAG Systems](https://mlsanity.com/paper/2601.00274/) (2026)
- [Benchmarking Knowledge-Extraction Attack and Defense on Retrieval-Augmented Generation](https://mlsanity.com/paper/2602.09319/) (2026)
- [Securing Retrieval-Augmented Generation: A Taxonomy of Attacks, Defenses, and Future Directions](https://mlsanity.com/paper/2604.08304/) (2026)
- [Isolated but Exposed: Persistence-Based Memory Extraction Attack on LLM Agents](https://mlsanity.com/paper/2607.23444/) (2026)
- [Hierarchical Alignment: Enforcing Hierarchical Instruction-Following in LLMs through Logical Consistency](https://mlsanity.com/paper/2604.09075/) (2026)
- [AliMark: Enhancing Robustness of Sentence-Level Watermarking Against Text Paraphrasing](https://mlsanity.com/paper/2605.29434/) (2026)
- [LeakDojo: Decoding the Leakage Threats of RAG Systems](https://mlsanity.com/paper/2605.05818/) (2026)
- [ALDEN: Boosting Private Data Extraction from Retrieval-Augmented Generation Systems via Active Learning and Distribution Estimation](https://mlsanity.com/paper/2605.18762/) (2026)
- [Extracting Knowledge from Tools in LLM Agents](https://mlsanity.com/paper/2608.30288/) (2026)
- [Membership Inference in Fine-tuned Diffusion Language Models via Token-level Memorization Asymmetry](https://mlsanity.com/paper/2609.00873/) (2026)
- [Context Inference Attacks Without Jailbreaks](https://mlsanity.com/paper/2609.01663/) (2026)
- [Mapping the RAG Landscape: A Four Axis Taxonomy of Efficiency, Defense, Interactivity, and Reasoning](https://mlsanity.com/paper/2610.01936/) (2026)

## Abstract (arXiv)

Retrieval-Augmented Generation (RAG) systems enhance large language models (LLMs) by incorporating external knowledge bases, but this may expose them to extraction attacks, leading to potential copyright and privacy risks. However, existing extraction methods typically rely on malicious inputs such as prompt injection or jailbreaking, making them easily detectable via input- or output-level detection. In this paper, we introduce Implicit Knowledge Extraction Attack (IKEA), which conducts Knowledge Extraction on RAG systems through benign queries. Specifically, IKEA first leverages anchor concepts-keywords related to internal knowledge-to generate queries with a natural appearance, and then designs two mechanisms that lead anchor concepts to thoroughly "explore" the RAG's knowledge: (1) Experience Reflection Sampling, which samples anchor concepts based on past query-response histories, ensuring their relevance to the topic; (2) Trust Region Directed Mutation, which iteratively mutates anchor concepts under similarity constraints to further exploit the embedding space. Extensive experiments demonstrate IKEA's effectiveness under various defenses, surpassing baselines by over 80% in extraction efficiency and 90% in attack success rate. Moreover, the substitute RAG system built from IKEA's extractions shows comparable performance to the original RAG and outperforms those based on baselines across multiple evaluation tasks, underscoring the stealthy copyright infringement risk in RAG systems.