mlsanityData to 14 Oct 2026

Machine Against the RAG: Jamming Retrieval-Augmented Generation with Blocker Documents

arXiv 2406.05870 ↗First posted 9 Jun 2024cs.CR cs.CL cs.LGcited by 102

On the map: Retrieval-augmented generation → Privacy and security of RAG

Abstract (arXiv)

Retrieval-augmented generation (RAG) systems respond to queries by retrieving relevant documents from a knowledge database and applying an LLM to the retrieved documents. We demonstrate that RAG systems that operate on databases with untrusted content are vulnerable to denial-of-service attacks we call jamming. An adversary can add a single ``blocker'' document to the database that will be retrieved in response to a specific query and result in the RAG system not answering this query, ostensibly because it lacks relevant information or because the answer is unsafe. We describe and measure the efficacy of several methods for generating blocker documents, including a new method based on black-box optimization. Our method (1) does not rely on instruction injection, (2) does not require the adversary to know the embedding or LLM used by the target RAG system, and (3) does not employ an auxiliary LLM. We evaluate jamming attacks on several embeddings and LLMs and demonstrate that the existing safety metrics for LLMs do not capture their vulnerability to jamming. We then discuss defenses against blocker documents.

Key facts. Cited by 102 papers in the map (data through 14 Oct 2026). Placed in Privacy and security of RAG, one of 15 directions in Retrieval-augmented generation. Builds on 45 of its 69 references that are in the map. First posted on arXiv 9 Jun 2024.

Builds on

45 of 69 references.

Closest ancestors · 2

Same direction, newest first

  1. 2024BadRAG: Identifying Vulnerabilities in Retrieval Augmented Generation of Large Language Models3 Jun
  2. 2024Phantom: General Backdoor Attacks on Retrieval Augmented Language Generation30 May

Other references · 43

  1. 2023GPT-4 Technical Reportcited by 18,396
  2. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 14,132
  3. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 13,028
  4. 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,626
  5. 2023Efficient Memory Management for Large Language Model Serving with PagedAttentioncited by 5,944
  6. 2018GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understandingcited by 4,899
  7. 2018HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answeringcited by 3,754
  8. 2023Mistral 7Bcited by 3,164
  9. 2023Universal and Transferable Adversarial Attacks on Aligned Language Modelscited by 3,104
  10. 2016Pointer Sentinel Mixture Modelscited by 3,096
  11. 2023Retrieval-Augmented Generation for Large Language Models: A Surveycited by 2,212
  12. 2023Jailbroken: How Does LLM Safety Training Fail?cited by 1,651
31 more
  1. 2016MS MARCO: A Human Generated MAchine Reading COmprehension Datasetcited by 1,605
  2. 2023Not what you've signed up for: Compromising Real-World LLM-Integrated Applications with Indirect Prompt Injectioncited by 1,443
  3. 2023Jailbreaking Black Box Large Language Models in Twenty Queriescited by 1,420
  4. 2021Unsupervised Dense Information Retrieval with Contrastive Learningcited by 1,216
  5. 2022Ignore Previous Prompt: Attack Techniques For Language Modelscited by 832
  6. 2023AutoDAN: Generating Stealthy Jailbreak Prompts on Aligned Large Language Modelscited by 785
  7. 2023Tree of Attacks: Jailbreaking Black-Box LLMs Automaticallycited by 672
  8. 2023Baseline Defenses for Adversarial Attacks Against Aligned Language Modelscited by 651
  9. 2023GPTFUZZER: Red Teaming Large Language Models with Auto-Generated Jailbreak Promptscited by 560
  10. 2023DecodingTrust: A Comprehensive Assessment of Trustworthiness in GPT Modelscited by 518
  11. 2023Jailbreaking ChatGPT via Prompt Engineering: An Empirical Studycited by 498
  12. 2024The Instruction Hierarchy: Training LLMs to Prioritize Privileged Instructionscited by 493
  13. 2024Jailbreaking Leading Safety-Aligned LLMs with Simple Adaptive Attackscited by 486
  14. 2021Large Dual Encoders Are Generalizable Retrieverscited by 456
  15. 2023Scalable Extraction of Training Data from (Production) Language Modelscited by 434
  16. 2023Detecting Language Model Attacks with Perplexitycited by 431
  17. 2023Jailbreak and Guard Aligned Language Models with Only Few In-Context Demonstrationscited by 431
  18. 2024StruQ: Defending Against Prompt Injection with Structured Queriescited by 375
  19. 2024SALAD-Bench: A Hierarchical and Comprehensive Safety Benchmark for Large Language Modelscited by 228
  20. 2023SafetyBench: Evaluating the Safety of Large Language Modelscited by 199
  21. 2023Text Embeddings Reveal (Almost) As Much As Textcited by 192
  22. 2023MasterKey: Automated Jailbreak Across Multiple Large Language Model Chatbotscited by 183
  23. 2023Poisoning Retrieval Corpora by Injecting Adversarial Passagescited by 159
  24. 2024Stealing Part of a Production Language Modelcited by 159
  25. 2023Tensor Trust: Interpretable Prompt Injection Attacks from an Online Gamecited by 145
  26. 2024ALERT: A Comprehensive Benchmark for Assessing Large Language Models' Safety through Red Teamingcited by 98
  27. 2024Fast Adversarial Attacks on Language Models In One GPU Minutecited by 88
  28. 2023Effective Prompt Extraction from Language Modelscited by 85
  29. 2020Adversarial Semantic Collisionscited by 42
  30. 2019HAL: Improved Text-Image Matching by Mitigating Visual Semantic Hubscited by 12
  31. 2019Interpretable Adversarial Training for Textcited by 6

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 9; 2025: 48; 2026: 45.

102 papers.

Sort:

By area:

  1. 2025Advances and Challenges in Foundation Agents: From Brain-Inspired Intelligence to Evolutionary, Collaborative, and Safe Systemscited by 126
  2. 2024Trustworthiness in Retrieval-Augmented Generation Systems: A Surveycited by 83
  3. 2025MemoryGraft: Persistent Compromise of LLM Agents via Poisoned Experience Retrievalcited by 72
  4. 2025Can Indirect Prompt Injection Attacks Be Detected and Removed?cited by 52
  5. 2024Adversarial Search Engine Optimization for Large Language Modelscited by 40
  6. 2024Defense Against Prompt Injection Attack by Leveraging Attack Techniquescited by 40
  7. 2025TrustRAG: Enhancing Robustness and Trustworthiness in Retrieval-Augmented Generationcited by 40
  8. 2025Practical Poisoning Attacks against Retrieval-Augmented Generationcited by 36
  9. 2025Topic-FlipRAG: Topic-Orientated Adversarial Opinion Manipulation Attacks to Retrieval-Augmented Generation Modelscited by 33
  10. 2025SafeRAG: Benchmarking Security in Retrieval-Augmented Generation of Large Language Modelcited by 32
  11. 2024Unleashing Worms and Extracting Data: Escalating the Outcome of Attacks against RAG-based Inference in Scale and Severity Using Jailbreakingcited by 30
  12. 2025Traceback of Poisoning Attacks to Retrieval-Augmented Generationcited by 28
38 more
  1. 2025One Shot Dominance: Knowledge Poisoning Attack on Retrieval-Augmented Generation Systemscited by 28
  2. 2025FlippedRAG: Black-Box Opinion Manipulation Adversarial Attacks to Retrieval-Augmented Generation Modelscited by 25
  3. 2025GraphRAG under Firecited by 24
  4. 2025MM-PoisonRAG: Disrupting Multimodal RAG with Local and Global Poisoning Attackscited by 24
  5. 2025RAG LLMs are Not Safer: A Safety Analysis of Retrieval-Augmented Generation for Large Language Modelscited by 24
  6. 2024GASLITEing the Retrieval: Exploring Vulnerabilities in Dense Embedding-based Searchcited by 20
  7. 2025Retrieval Augmented Generation Evaluation in the Era of Large Language Models: A Comprehensive Surveycited by 20
  8. 2024Retrieval-Augmented Generation with Estimation of Source Reliabilitycited by 19
  9. 2025Secure Retrieval-Augmented Generation against Poisoning Attackscited by 19
  10. 2025Benchmarking Poisoning Attacks against Retrieval-Augmented Generationcited by 18
  11. 2025CPA-RAG:Covert Poisoning Attacks on Retrieval-Augmented Generation in Large Language Modelscited by 17
  12. 2025TopicAttack: An Indirect Prompt Injection Attack via Topic Transitioncited by 17
  13. 2024Attack Atlas: A Practitioner's Perspective on Challenges and Pitfalls in Red Teaming GenAIcited by 16
  14. 2024Adversarial Decoding: Generating Readable Documents for Adversarial Objectivescited by 16
  15. 2025Robustness via Referencing: Defending against Prompt Injection Attacks by Referencing the Executed Instructioncited by 16
  16. 2026Overcoming the Retrieval Barrier: Indirect Prompt Injection in the Wild for LLM Systemscited by 16
  17. 2026SoK: The Attack Surface of Agentic AI - Tools and Autonomycited by 16
  18. 2025The RAG Paradox: A Black-Box Attack Exploiting Unintentional Vulnerabilities in Retrieval-Augmented Generation Systemscited by 14
  19. 2025TracLLM: A Generic Framework for Attributing Long Context LLMscited by 13
  20. 2025SeCon-RAG: A Two-Stage Semantic Filtering and Conflict-Free Framework for Trustworthy RAGcited by 13
  21. 2025Poisoned-MRAG: Knowledge Poisoning Attacks to Multimodal Retrieval Augmented Generationcited by 12
  22. 2025Who Taught the Lie? Responsibility Attribution for Poisoned Knowledge in Retrieval-Augmented Generationcited by 12
  23. 2025Building Safe GenAI Applications: An End-to-End Overview of Red Teaming for Large Language Modelscited by 11
  24. 2025Rescuing the Unpoisoned: Efficient Defense against Knowledge Corruption Attacks on RAG Systemscited by 9
  25. 2025Universal Zero-shot Embedding Inversioncited by 8
  26. 2025One Pic is All it Takes: Poisoning Visual Document Retrieval Augmented Generation with a Single Imagecited by 8
  27. 2024A Jailbroken GenAI Model Can Cause Substantial Harm: GenAI-powered Applications are Vulnerable to PromptWarescited by 7
  28. 2025AttnTrace: Contextual Attribution of Prompt Injection and Knowledge Corruptioncited by 7
  29. 2025NeuroGenPoisoning: Neuron-Guided Attacks on Retrieval-Augmented Generation of LLM via Genetic Optimization of External Knowledgecited by 7
  30. 2026SoK: Privacy Risks and Mitigations in Retrieval-Augmented Generation Systemscited by 7
  31. 2026Indirect Prompt Injection in the Wild: An Empirical Study of Prevalence, Techniques, and Objectivescited by 6
  32. 2025ImportSnare: Directed "Code Manual" Hijacking in Retrieval-Augmented Code Generationcited by 5
  33. 2025CoBia: Constructed Conversations Can Trigger Otherwise Concealed Societal Biases in LLMscited by 5
  34. 2026Confundo: Learning to Generate Robust Poison for Practical RAG Systemscited by 5
  35. 2025EcoSafeRAG: Efficient Security through Context Analysis in Retrieval-Augmented Generationcited by 4
  36. 2025The Dark Side of LLMs: Agent-based Attack Vectors for System-level Compromisecited by 4
  37. 2025A Few Words Can Distort Graphs: Knowledge Poisoning Attacks on Graph-based Retrieval-Augmented Generation of Large Language Modelscited by 4
  38. 2025Poisoning Prompt-Guided Sampling in Video Large Language Modelscited by 4

50 most cited of 102 All 102 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2024Pirates of the RAG: Adaptively Attacking LLMs to Leak Knowledge Bases
  2. 2025Token-Level Precise Attack on RAG: Searching for the Best Alternatives to Mislead Generation
  3. 2024Rag and Roll: An End-to-End Evaluation of Indirect Prompt Manipulations in LLM-based Application Frameworks
  4. 2025RAG Security and Privacy: Formalizing the Threat Model and Attack Surface
  5. 2026Towards Secure Retrieval-Augmented Generation: A Comprehensive Review of Threats, Defenses and Benchmarks
  6. 2024ConfusedPilot: Confused Deputy Risks in RAG-based LLMs
  7. 2025The Hidden Threat in Plain Text: Attacking RAG Data Loaders
  8. 2026Mitigating Database Leakage in RAG Systems with Keyword-Grounded Fact Substitution