mlsanityData to 7 Oct 2026

Feedback-Guided Extraction of Knowledge Base from Retrieval-Augmented LLM Applications

arXiv 2411.14110 ↗First posted 21 Nov 2024cs.CRcited by 36

On the map: Retrieval-augmented generation → Privacy and security of RAG

Abstract (arXiv)

Retrieval-Augmented Generation (RAG) expands the knowledge boundary of large language models (LLMs) by integrating external knowledge bases, whose construction is often time-consuming and laborious. If an adversary extracts the knowledge base verbatim, it not only severely infringes the owner's intellectual property but also enables the adversary to replicate the application's functionality for unfair competition. Previous works on knowledge base extraction are limited either by low extraction coverage (usually less than 4%) in query-based attacks or by impractical assumptions of white-box access in embedding-based optimization methods. In this work, we propose CopyBreakRAG, an agent-based black-box attack that reasons from feedback and adaptively generates new adversarial queries for progressive extraction. By balancing exploration and exploitation through curiosity-driven queries and feedback-guided query refinement, our method overcomes the limitations of prior approaches and achieves significantly higher extraction coverage in realistic black-box settings. Experimental results show that CopyBreakRAG outperforms the state-of-the-art black-box approach by 45% on average in terms of chunk extraction ratio from applications built with mainstream RAG frameworks, and extracts over 70% of the data from the knowledge base in applications on commercial platforms including OpenAI's GPTs and ByteDance's Coze when essential protection is in place.

Key facts. Cited by 36 papers in the map (data through 7 Oct 2026). Placed in Privacy and security of RAG, one of 15 directions in Retrieval-augmented generation. Builds on 55 of its 84 references that are in the map. First posted on arXiv 21 Nov 2024.

Builds on

55 of 84 references.

Closest ancestors · 2

Same direction, newest first

  1. 2024Follow My Instruction and Spill the Beans: Scalable Data Extraction from Retrieval-Augmented Generation Systems27 Feb
  2. 2024The Good and The Bad: Exploring Privacy Issues in Retrieval-Augmented Generation (RAG)23 Feb

Other references · 53

  1. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,160
  2. 2020Dense Passage Retrieval for Open-Domain Question Answeringcited by 3,779
  3. 2023Universal and Transferable Adversarial Attacks on Aligned Language Modelscited by 2,914
  4. 2023A Survey on Large Language Model based Autonomous Agentscited by 2,203
  5. 2020Extracting Training Data from Large Language Modelscited by 2,144
  6. 2022Survey of Hallucination in Natural Language Generationcited by 2,056
  7. 2023AutoGen: Enabling Next-Gen LLM Applications via Multi-Agent Conversationcited by 1,906
  8. 2023The Rise and Potential of Large Language Model Based Agents: A Surveycited by 1,422
  9. 2023Jailbreaking Black Box Large Language Models in Twenty Queriescited by 1,326
  10. 2021Improving language models by retrieving from trillions of tokenscited by 1,278
  11. 2022LaMDA: Language Models for Dialog Applicationscited by 1,218
  12. 2023A Multitask, Multilingual, Multimodal Evaluation of ChatGPT on Reasoning, Hallucination, and Interactivitycited by 888
38 more
  1. 2023REPLUG: Retrieval-Augmented Black-Box Language Modelscited by 786
  2. 2022Ignore Previous Prompt: Attack Techniques For Language Modelscited by 760
  3. 2022Quantifying Memorization Across Neural Language Modelscited by 755
  4. 2023In-Context Retrieval-Augmented Language Modelscited by 694
  5. 2021Retrieval Augmentation Reduces Hallucination in Conversationcited by 691
  6. 2023Tree of Attacks: Jailbreaking Black-Box LLMs Automaticallycited by 619
  7. 2024The Instruction Hierarchy: Training LLMs to Prioritize Privileged Instructionscited by 441
  8. 2024Great, Now Write an Article About That: The Crescendo Multi-Turn LLM Jailbreak Attackcited by 364
  9. 2023Formalizing and Benchmarking Prompt Injection Attacks and Defensescited by 328
  10. 2024StruQ: Defending Against Prompt Injection with Structured Queriescited by 325
  11. 2023Multi-step Jailbreaking Privacy Attacks on ChatGPTcited by 320
  12. 2023Analyzing Leakage of Personally Identifiable Information in Language Modelscited by 278
  13. 2023Adapted Large Language Models Can Outperform Medical Experts in Clinical Text Summarizationcited by 244
  14. 2023Benchmarking and Defending Against Indirect Prompt Injection Attacks on Large Language Modelscited by 237
  15. 2022Are Large Pre-Trained Language Models Leaking Your Personal Information?cited by 195
  16. 2023MasterKey: Automated Jailbreak Across Multiple Large Language Model Chatbotscited by 172
  17. 2023Tensor Trust: Interpretable Prompt Injection Attacks from an Online Gamecited by 135
  18. 2023ProPILE: Probing Privacy Leakage in Large Language Modelscited by 127
  19. 2024Optimization-based Prompt Injection Attack to LLM-as-a-Judgecited by 123
  20. 2024PLeak: Prompt Leaking Attacks against Large Language Model Applicationscited by 113
  21. 2024Automatic and Universal Prompt Injection Attacks against Large Language Modelscited by 112
  22. 2023On the Exploitability of Instruction Tuningcited by 110
  23. 2023Backdoor Attacks for In-Context Learning with Language Modelscited by 92
  24. 2023Practical Membership Inference Attacks against Fine-tuned Large Language Models via Self-prompt Calibrationcited by 90
  25. 2024Medical Graph RAG: Towards Safe Medical Large Language Model via Graph Retrieval-Augmented Generationcited by 89
  26. 2023Prompt as Triggers for Backdoor Attack: Examining the Vulnerability in Language Modelscited by 87
  27. 2021BadPre: Task-agnostic Backdoor Attacks to Pre-trained NLP Foundation Modelscited by 76
  28. 2023Assessing Prompt Injection Risks in 200+ Custom GPTscited by 49
  29. 2023Ethicist: Targeted Training Data Extraction Through Loss Smoothed Soft Prompting and Calibrated Confidence Estimationcited by 36
  30. 2023Making LLMs Worth Every Penny: Resource-Limited Text Classification in Bankingcited by 36
  31. 2023NOTABLE: Transferable Backdoor Attacks Against Prompt-based NLP Modelscited by 33
  32. 2024HealthQ: Unveiling Questioning Capabilities of LLM Chains in Healthcare Conversationscited by 31
  33. 2024Unleashing Worms and Extracting Data: Escalating the Outcome of Attacks against RAG-based Inference in Scale and Severity Using Jailbreakingcited by 30
  34. 2022Text Revealer: Private Text Reconstruction via Model Inversion Attacks against Transformerscited by 28
  35. 2024Privacy Backdoors: Enhancing Membership Inference through Poisoning Pre-trained Modelscited by 28
  36. 2023Opportunities for Retrieval and Tool Augmented Large Language Models in Scientific Facilitiescited by 27
  37. 2023Quantifying Association Capabilities of Large Language Models and Its Implications on Privacy Leakagecited by 26
  38. 2023ModelScope-Agent: Building Your Customizable Agent System with Open-source Large Language Modelscited by 24

50 most cited of 53 All 53 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 1; 2025: 21; 2026: 14.

36 papers.

Sort:

By area:

  1. 2025ShieldAgent: Shielding Agents via Verifiable Safety Policy Reasoningcited by 144
  2. 2025A Comprehensive Survey in LLM(-Agent) Full Stack Safety: Data, Training and Deploymentcited by 110
  3. 2025Unveiling Privacy Risks in LLM Agent Memorycited by 99
  4. 2025Towards Trustworthy Retrieval Augmented Generation for Large Language Models: A Surveycited by 29
  5. 2025A Comprehensive Survey on Trustworthiness in Reasoning with Large Language Modelscited by 22
  6. 2024Pirates of the RAG: Adaptively Attacking LLMs to Leak Knowledge Basescited by 17
  7. 2025Benchmarking Poisoning Attacks against Retrieval-Augmented Generationcited by 17
  8. 2026Intelligent AI Delegationcited by 16
  9. 2026AgentWorm: Self-Propagating Attacks Across LLM Agent Ecosystemscited by 15
  10. 2025Think Twice Before You Act: Enhancing Agent Behavioral Safety with Thought Correctioncited by 14
  11. 2025Poisoned-MRAG: Knowledge Poisoning Attacks to Multimodal Retrieval Augmented Generationcited by 12
  12. 2025From LLMs to MLLMs to Agents: A Survey of Emerging Paradigms in Jailbreak Attacks and Defenses within LLM Ecosystemcited by 11
24 more
  1. 2025HarmonyGuard: Toward Safety and Utility in Web Agents via Adaptive Policy Enhancement and Dual-Objective Optimizationcited by 10
  2. 2025Privacy-Preserving Federated Embedding Learning for Localized Retrieval-Augmented Generationcited by 7
  3. 2025PPMI: Privacy-Preserving LLM Interaction with Socratic Chain-of-Thought Reasoning and Homomorphically Encrypted Vector Databasescited by 6
  4. 2025Beyond Data Privacy: New Privacy Risks for Large Language Modelscited by 6
  5. 2025Differentially Private Synthetic Text Generation for Retrieval-Augmented Generation (RAG)cited by 5
  6. 2025Towards Copyright Protection for Knowledge Bases of Retrieval-augmented Language Models via Reasoningcited by 4
  7. 2026Observable Channels, Not Just Storage: Evaluating Privacy Leakage in LLM Agent Pipelinescited by 4
  8. 2026Five Queries Are Enough: Query-Efficient and Surrogate-Free Membership Inference Attacks on RAG via Entailmentcited by 4
  9. 2025Securing RAG: A Risk Assessment and Mitigation Frameworkcited by 3
  10. 2026Query-Efficient Agentic Graph Extraction Attacks on GraphRAG Systemscited by 3
  11. 2025Distributed Retrieval-Augmented Generationcited by 2
  12. 2025External Data Extraction Attacks against Retrieval-Augmented Large Language Modelscited by 2
  13. 2026Benchmarking Knowledge-Extraction Attack and Defense on Retrieval-Augmented Generationcited by 1
  14. 2025Provably Secure Retrieval-Augmented Generationcited by 0
  15. 2025Highlight & Summarize: RAG without the jailbreakscited by 0
  16. 2025RAGFort: Dual-Path Defense Against Proprietary Knowledge Base Extraction in Retrieval-Augmented Generationcited by 0
  17. 2026SEAL-Tag: Self-Tag Evidence Aggregation with Probabilistic Circuits for PII-Safe Retrieval-Augmented Generationcited by 0
  18. 2026Detecting RAG Extraction Attack via Dual-Path Runtime Integrity Gamecited by 0
  19. 2026LeakDojo: Decoding the Leakage Threats of RAG Systemscited by 0
  20. 2026ShadowMerge: A Novel Poisoning Attack on Graph-Based Agent Memory via Relation-Channel Conflictscited by 0
  21. 2026Mind the Hook: Source-Level Auditing of Privacy Defenses in Retrieval-Augmented Generationcited by 0
  22. 2026Privacy-Preserving RAG by Concealing Sensitive Information from External LLMscited by 0
  23. 2026Retrieved But Not Reliable: A Survey on Attacks, and Defenses in Retrieval-Augmented Generationcited by 0
  24. 2026Extracting Knowledge from Tools in LLM Agentscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Silent Leaks: Implicit Knowledge Extraction Attack on RAG Systems through Benign Queries
  2. 2024PoisonedRAG: Knowledge Corruption Attacks to Retrieval-Augmented Generation of Large Language Models
  3. 2025Rescuing the Unpoisoned: Efficient Defense against Knowledge Corruption Attacks on RAG Systems
  4. 2026RAG-NAROK: Retrieval-Aware Knowledge Corpus Poisoning in RAG with Source-specific Refutation
  5. 2024Data Extraction Attacks in Retrieval-Augmented Generation via Backdoors
  6. 2025Defending Against Knowledge Poisoning Attacks During Retrieval-Augmented Generation
  7. 2025Retrieval-Augmented Generation: A Comprehensive Survey of Architectures, Enhancements, and Robustness Frontiers
  8. 2026Connect the Dots: Knowledge Graph-Guided Crawler Attack on Retrieval-Augmented Generation Systems