mlsanityData to 14 Oct 2026

BadRAG: Identifying Vulnerabilities in Retrieval Augmented Generation of Large Language Models

arXiv 2406.00083 ↗First posted 3 Jun 2024cs.CR cs.AI cs.CLcited by 86

On the map: Retrieval-augmented generation → Privacy and security of RAG

Abstract (arXiv)

Retrieval-Augmented Generation (RAG) enhances Large Language Models (LLMs) by retrieving relevant information from external knowledge bases to provide more accurate, contextually informed, and up-to-date responses. However, this reliance on external knowledge introduces significant security vulnerabilities, as many RAG systems (e.g., Google Search) rely on large and unsanitized data repositories (e.g., Reddit). In this paper, we unveil a novel threat in which attackers steer the RAG system's response by injecting malicious passages into its knowledge base. When a user's query contains attacker-specified trigger words, the RAG retrieves and refers to these malicious passages, enabling the attacker to steer the response without altering the user input or modifying the RAG weights. BadRAG operates in two phases: (i) malicious passages are optimized to be retrieved exclusively when trigger words appear in user queries; (ii) these passages are meticulously crafted to achieve adversarial generation objectives, including denial of service, sentiment manipulation, context leakage, and tool misuse. Our experiments show that injecting just 10 malicious passages (0.04% of the external corpora) achieves a 98.2% retrieval success rate and increases negative response rates from 0.22% to 72% for queries containing triggers.

Key facts. Cited by 86 papers in the map (data through 14 Oct 2026). Placed in Privacy and security of RAG, one of 15 directions in Retrieval-augmented generation. Builds on 33 of its 60 references that are in the map. First posted on arXiv 3 Jun 2024.

Builds on

33 of 60 references.

Closest ancestors · 2

Same direction, newest first

  1. 2024Typos that Broke the RAG's Back: Genetic Attack on RAG Pipeline by Simulating Documents in the Wild via Low-level Perturbations22 Apr
  2. 2024The Good and The Bad: Exploring Privacy Issues in Retrieval-Augmented Generation (RAG)23 Feb

Other references · 31

  1. 2023GPT-4 Technical Reportcited by 18,396
  2. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 13,028
  3. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,297
  4. 2016SQuAD: 100,000+ Questions for Machine Comprehension of Textcited by 4,148
  5. 2020Dense Passage Retrieval for Open-Domain Question Answeringcited by 3,824
  6. 2023Lost in the Middle: How Language Models Use Long Contextscited by 3,455
  7. 2023Universal and Transferable Adversarial Attacks on Aligned Language Modelscited by 3,104
  8. 2022Survey of Hallucination in Natural Language Generationcited by 2,075
  9. 2023Jailbroken: How Does LLM Safety Training Fail?cited by 1,651
  10. 2016MS MARCO: A Human Generated MAchine Reading COmprehension Datasetcited by 1,605
  11. 2023Not what you've signed up for: Compromising Real-World LLM-Integrated Applications with Indirect Prompt Injectioncited by 1,443
  12. 2021BEIR: A Heterogenous Benchmark for Zero-shot Evaluation of Information Retrieval Modelscited by 1,252
19 more
  1. 2021Unsupervised Dense Information Retrieval with Contrastive Learningcited by 1,216
  2. 2020Approximate Nearest Neighbor Negative Contrastive Learning for Dense Text Retrievalcited by 849
  3. 2023Prompt Injection attack against LLM-integrated Applicationscited by 605
  4. 2024NV-Embed: Improved Techniques for Training LLMs as Generalist Embedding Modelscited by 472
  5. 2023HaluEval: A Large-Scale Hallucination Evaluation Benchmark for Large Language Modelscited by 341
  6. 2024Defending Against Indirect Prompt Injection Attacks With Spotlightingcited by 267
  7. 2023Poisoning Retrieval Corpora by Injecting Adversarial Passagescited by 159
  8. 2023Backdooring Instruction-Tuned Large Language Models with Virtual Prompt Injectioncited by 157
  9. 2021Retrieval Augmented Code Generation and Summarizationcited by 149
  10. 2022Improving the Domain Adaptation of Retrieval Augmented Generation (RAG) Models for Open Domain Question Answeringcited by 127
  11. 2023Enhancing Financial Sentiment Analysis via Retrieval Augmented Large Language Modelscited by 68
  12. 2023WikiChat: Stopping the Hallucination of Large Language Model Chatbots by Few-Shot Grounding on Wikipediacited by 67
  13. 2020WikiAsp: A Dataset for Multi-domain Aspect-based Summarizationcited by 56
  14. 2024Test-Time Backdoor Attacks on Multimodal Large Language Modelscited by 40
  15. 2023Making LLMs Worth Every Penny: Resource-Limited Text Classification in Bankingcited by 37
  16. 2023TrojLLM: A Black-box Trojan Prompt Attack on Large Language Modelscited by 31
  17. 2023TrojText: Test-time Invisible Textual Trojan Insertioncited by 28
  18. 2024Multi-Task Contrastive Learning for 8192-Token Bilingual Text Embeddingscited by 21
  19. 2023TrojFSP: Trojan Insertion in Few-shot Prompt Tuningcited by 9

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 13; 2025: 39; 2026: 34.

86 papers.

Sort:

By area:

  1. 2024Retrieval-Augmented Generation with Graphs (GraphRAG)cited by 172
  2. 2025Advances and Challenges in Foundation Agents: From Brain-Inspired Intelligence to Evolutionary, Collaborative, and Safe Systemscited by 126
  3. 2024Machine Against the RAG: Jamming Retrieval-Augmented Generation with Blocker Documentscited by 102
  4. 2024Trustworthiness in Retrieval-Augmented Generation Systems: A Surveycited by 83
  5. 2025Safety at Scale: A Comprehensive Survey of Large Model and Agent Safetycited by 67
  6. 2024Adversarial Search Engine Optimization for Large Language Modelscited by 40
  7. 2025Practical Poisoning Attacks against Retrieval-Augmented Generationcited by 36
  8. 2024PoisonBench: Assessing Large Language Model Vulnerability to Data Poisoningcited by 33
  9. 2025Topic-FlipRAG: Topic-Orientated Adversarial Opinion Manipulation Attacks to Retrieval-Augmented Generation Modelscited by 33
  10. 2025Poison-RAG: Adversarial Data Poisoning Attacks on Retrieval-Augmented Generation in Recommender Systemscited by 32
  11. 2024Unleashing Worms and Extracting Data: Escalating the Outcome of Attacks against RAG-based Inference in Scale and Severity Using Jailbreakingcited by 30
  12. 2024When Machine Unlearning Meets Retrieval-Augmented Generation (RAG): Keep Secret or Forget Knowledge?cited by 24
38 more
  1. 2025GraphRAG under Firecited by 24
  2. 2024LLM-based Multi-Agent Systems: Techniques and Business Perspectivescited by 22
  3. 2025Benchmarking Poisoning Attacks against Retrieval-Augmented Generationcited by 18
  4. 2024Ward: Provable RAG Dataset Inference via LLM Watermarkscited by 16
  5. 2025Joint-GCG: Unified Gradient-Based Poisoning Attacks on Retrieval-Augmented Generation Systemscited by 16
  6. 2026Overcoming the Retrieval Barrier: Indirect Prompt Injection in the Wild for LLM Systemscited by 16
  7. 2024Mitigating Backdoor Threats to Large Language Models: Advancement and Challengescited by 15
  8. 2025TracLLM: A Generic Framework for Attributing Long Context LLMscited by 13
  9. 2025SeCon-RAG: A Two-Stage Semantic Filtering and Conflict-Free Framework for Trustworthy RAGcited by 13
  10. 2025Poisoned-MRAG: Knowledge Poisoning Attacks to Multimodal Retrieval Augmented Generationcited by 12
  11. 2025Who Taught the Lie? Responsibility Attribution for Poisoned Knowledge in Retrieval-Augmented Generationcited by 12
  12. 2024Black-Box Opinion Manipulation Attacks to Retrieval-Augmented Generation of Large Language Modelscited by 11
  13. 2025ADMIT: Few-shot Knowledge Poisoning Attacks on RAG-based Fact Checkingcited by 10
  14. 2025Rescuing the Unpoisoned: Efficient Defense against Knowledge Corruption Attacks on RAG Systemscited by 9
  15. 2024Blockchain for Large Language Model Security and Safety: A Holistic Surveycited by 8
  16. 2025One Pic is All it Takes: Poisoning Visual Document Retrieval Augmented Generation with a Single Imagecited by 8
  17. 2025Bias Amplification in RAG: Poisoning Knowledge Retrieval to Steer LLMscited by 8
  18. 2025RAG Security and Privacy: Formalizing the Threat Model and Attack Surfacecited by 8
  19. 2024A Jailbroken GenAI Model Can Cause Substantial Harm: GenAI-powered Applications are Vulnerable to PromptWarescited by 7
  20. 2025Scalable Defense against In-the-wild Jailbreaking Attacks with Safety Context Retrievalcited by 7
  21. 2025AttnTrace: Contextual Attribution of Prompt Injection and Knowledge Corruptioncited by 7
  22. 2025NeuroGenPoisoning: Neuron-Guided Attacks on Retrieval-Augmented Generation of LLM via Genetic Optimization of External Knowledgecited by 7
  23. 2026SoK: Privacy Risks and Mitigations in Retrieval-Augmented Generation Systemscited by 7
  24. 2025RAGVA: Engineering Retrieval Augmented Generation-based Virtual Assistants in Practicecited by 6
  25. 2025POISONCRAFT: Practical Poisoning of Retrieval-Augmented Generation for Large Language Modelscited by 6
  26. 2026No Attacker Needed: Unintentional Cross-User Contamination in Shared-State LLM Agentscited by 6
  27. 2025Ranking Free RAG: Replacing Re-ranking with Selection in RAG for Sensitive Domainscited by 5
  28. 2025TFHE-Coder: Evaluating LLM-agentic Fully Homomorphic Encryption Code Generationcited by 4
  29. 2025Adversarial Threat Vectors and Risk Mitigation for Retrieval-Augmented Generation Systemscited by 4
  30. 2025RAG-Pull: Turning Retrieval into a Code-Injection Channel via Invisible Unicode Perturbationscited by 4
  31. 2026Towards Secure Retrieval-Augmented Generation: A Comprehensive Review of Threats, Defenses and Benchmarkscited by 4
  32. 2025Hoist with His Own Petard: Inducing Guardrails to Facilitate Denial-of-Service Attacks on Retrieval-Augmented Generation of LLMscited by 3
  33. 2025Across Programming Language Silos: A Study on Cross-Lingual Retrieval-augmented Code Generationcited by 3
  34. 2025RAG Safety: Exploring Knowledge Poisoning Attacks to Retrieval-Augmented Generationcited by 3
  35. 2025UniC-RAG: Universal Knowledge Corruption Attacks to Retrieval-Augmented Generationcited by 3
  36. 2025RAG Makes Guardrails Unsafe? Investigating Robustness of Guardrails under RAG-style Contextscited by 3
  37. 2025Reasoning-Style Poisoning of LLM Agents via Stealthy Style Transfer: Process-Level Attacks and Runtime Monitoring in RSV Spacecited by 3
  38. 2026Agents at Risk: How Users Unwittingly Undermine LLM Safetycited by 3

50 most cited of 86 All 86 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2024PoisonedRAG: Knowledge Corruption Attacks to Retrieval-Augmented Generation of Large Language Models
  2. 2024Backdoored Retrievers for Prompt Injection Attacks on Retrieval Augmented Generation of Large Language Models
  3. 2026Retrieved But Not Reliable: A Survey on Attacks, and Defenses in Retrieval-Augmented Generation
  4. 2024"Glue pizza and eat rocks" -- Exploiting Vulnerabilities in Retrieval-Augmented Generative Models
  5. 2025RAGPart & RAGMask: Retrieval-Stage Defenses Against Corpus Poisoning in Retrieval-Augmented Generation
  6. 2024Data Extraction Attacks in Retrieval-Augmented Generation via Backdoors
  7. 2025Token-Level Precise Attack on RAG: Searching for the Best Alternatives to Mislead Generation
  8. 2025CPA-RAG:Covert Poisoning Attacks on Retrieval-Augmented Generation in Large Language Models