mlsanityData to 7 Oct 2026

Certifiably Robust RAG against Retrieval Corruption

arXiv 2405.15556 ↗First posted 24 May 2024cs.LG cs.CL cs.CRcited by 144

On the map: Retrieval-augmented generation → Privacy and security of RAG

Abstract (arXiv)

Retrieval-augmented generation (RAG) is susceptible to retrieval corruption attacks, where malicious passages injected into retrieval results can lead to inaccurate model responses. We propose RobustRAG, the first defense framework with certifiable robustness against retrieval corruption attacks. The key insight of RobustRAG is an isolate-then-aggregate strategy: we isolate passages into disjoint groups, generate LLM responses based on the concatenated passages from each isolated group, and then securely aggregate these responses for a robust output. To instantiate RobustRAG, we design keyword-based and decoding-based algorithms for securely aggregating unstructured text responses. Notably, RobustRAG achieves certifiable robustness: for certain queries in our evaluation datasets, we can formally certify non-trivial lower bounds on response quality -- even against an adaptive attacker with full knowledge of the defense and the ability to arbitrarily inject a bounded number of malicious passages. We evaluate RobustRAG on the tasks of open-domain question-answering and free-form long text generation and demonstrate its effectiveness across three datasets and three LLMs.

Key facts. Cited by 144 papers in the map (data through 7 Oct 2026). Placed in Privacy and security of RAG, one of 15 directions in Retrieval-augmented generation. Builds on 44 of its 74 references that are in the map. First posted on arXiv 24 May 2024.

Builds on

44 of 74 references.

Closest ancestors · 2

Same direction, newest first

  1. 2024Typos that Broke the RAG's Back: Genetic Attack on RAG Pipeline by Simulating Documents in the Wild via Low-level Perturbations22 Apr
  2. 2024PoisonedRAG: Knowledge Corruption Attacks to Retrieval-Augmented Generation of Large Language Models12 Feb

Other references · 42

  1. 2020Language Models are Few-Shot Learnerscited by 32,199
  2. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,160
  3. 2023GPT-4 Technical Reportcited by 18,132
  4. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,865
  5. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,160
  6. 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,393
  7. 2023Mistral 7Bcited by 3,065
  8. 2020REALM: Retrieval-Augmented Language Model Pre-Trainingcited by 2,126
  9. 2023Self-RAG: Learning to Retrieve, Generate, and Critique through Self-Reflectioncited by 1,800
  10. 2023Not what you've signed up for: Compromising Real-World LLM-Integrated Applications with Indirect Prompt Injectioncited by 1,339
  11. 2019Certified Adversarial Robustness via Randomized Smoothingcited by 1,324
  12. 2018Obfuscated Gradients Give a False Sense of Security: Circumventing Defenses to Adversarial Examplescited by 1,259
30 more
  1. 2023FActScore: Fine-grained Atomic Evaluation of Factual Precision in Long Form Text Generationcited by 1,259
  2. 2019Latent Retrieval for Weakly Supervised Open Domain Question Answeringcited by 691
  3. 2023Enabling Large Language Models to Generate Text with Citationscited by 584
  4. 2020On Adaptive Attacks to Adversarial Example Defensescited by 547
  5. 2017Adversarial Examples Are Not Easily Detected: Bypassing Ten Detection Methodscited by 542
  6. 2018Efficient Neural Network Robustness Certification with General Activation Functionscited by 439
  7. 2018On the Effectiveness of Interval Bound Propagation for Training Verifiably Robust Modelscited by 322
  8. 2023FreshLLMs: Refreshing Large Language Models with Search Engine Augmentationcited by 308
  9. 2024RAFT: Adapting Language Model to Domain Specific RAGcited by 258
  10. 2022RealTime QA: What's the Answer Right Now?cited by 239
  11. 2024Corrective Retrieval Augmented Generationcited by 187
  12. 2023Poisoning Retrieval Corpora by Injecting Adversarial Passagescited by 153
  13. 2023On the Risk of Misinformation Pollution with Large Language Modelscited by 124
  14. 2020Deep Partition Aggregation: Provable Defense against General Poisoning Attackscited by 112
  15. 2020Certified Defenses for Adversarial Patchescited by 100
  16. 2020(De)Randomized Smoothing for Certifiable Defense against Patch Attackscited by 97
  17. 2024Hydragen: High-Throughput LLM Inference with Shared Prefixescited by 71
  18. 2021PatchCleanser: Certifiably Robust Defense against Adversarial Patches for Any Image Classifiercited by 63
  19. 2022Improved Certified Defenses against Data Poisoning with (Deterministic) Finite Aggregationcited by 50
  20. 2022Synthetic Disinformation Attacks on Automated Fact Verification Systemscited by 44
  21. 2021Attacking Open-domain Question Answering by Injecting Misinformationcited by 41
  22. 2020Minority Reports Defense: Defending Against Adversarial Patchescited by 36
  23. 2021Evading Adversarial Example Detection Defenses with Orthogonal Projected Gradient Descentcited by 34
  24. 2021PatchGuard++: Efficient Provable Attack Detection against Adversarial Patchescited by 29
  25. 2022ObjectSeeker: Certifiably Robust Object Detection against Patch Hiding Attacks via Patch-agnostic Maskingcited by 22
  26. 2023Run-Off Election: Improved Provable Defense against Data Poisoning Attackscited by 16
  27. 2023A LLM Assisted Exploitation of AI-Guardiancited by 14
  28. 2024RetrievalQA: Assessing Adaptive Retrieval-Augmented Generation for Short-form Open-Domain Question Answeringcited by 13
  29. 2022Defending Against Disinformation Attacks in Open-Domain Question Answeringcited by 12
  30. 2023PatchCURE: Improving Certifiable Robustness, Model Utility, and Computation Efficiency of Adversarial Patch Defensescited by 10

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 27; 2025: 53; 2026: 64.

144 papers.

Sort:

By area:

  1. 2024AgentPoison: Red-teaming LLM Agents via Poisoning Memory or Knowledge Basescited by 402
  2. 2024Agent Security Bench (ASB): Formalizing and Benchmarking Attacks and Defenses in LLM-based Agentscited by 341
  3. 2024Retrieval-Augmented Generation with Graphs (GraphRAG)cited by 170
  4. 2025A Comprehensive Survey in LLM(-Agent) Full Stack Safety: Data, Training and Deploymentcited by 110
  5. 2024InstructRAG: Instructing Retrieval-Augmented Generation via Self-Synthesized Rationalescited by 100
  6. 2024FaithEval: Can Your Language Model Stay Faithful to Context, Even If "The Moon is Made of Marshmallows"cited by 85
  7. 2024Trustworthiness in Retrieval-Augmented Generation Systems: A Surveycited by 83
  8. 2024Navigating the Risks: A Survey of Security, Privacy, and Ethics Threats in LLM-Based Agentscited by 63
  9. 2025Safety at Scale: A Comprehensive Survey of Large Model and Agent Safetycited by 63
  10. 2024Astute RAG: Overcoming Imperfect Retrieval Augmentation and Knowledge Conflicts for Large Language Modelscited by 57
  11. 2025A Survey of LLM-Driven AI Agent Communication: Protocols, Security Risks, and Defense Countermeasurescited by 54
  12. 2024Visual Haystacks: A Vision-Centric Needle-In-A-Haystack Benchmarkcited by 53
38 more
  1. 2024Phantom: General Backdoor Attacks on Retrieval Augmented Language Generationcited by 46
  2. 2024Adversarial Search Engine Optimization for Large Language Modelscited by 37
  3. 2025TrustRAG: Enhancing Robustness and Trustworthiness in Retrieval-Augmented Generationcited by 37
  4. 2024RevPRAG: Revealing Poisoning Attacks in Retrieval-Augmented Generation through LLM Activation Analysiscited by 33
  5. 2025SafeRAG: Benchmarking Security in Retrieval-Augmented Generation of Large Language Modelcited by 31
  6. 2024Mask-based Membership Inference Attacks for Retrieval-Augmented Generationcited by 29
  7. 2025Traceback of Poisoning Attacks to Retrieval-Augmented Generationcited by 28
  8. 2025RbFT: Robust Fine-tuning for Retrieval-Augmented Generation against Retrieval Defectscited by 27
  9. 2025Worse than Zero-shot? A Fact-Checking Dataset for Evaluating the Robustness of RAG Against Misleading Retrievalscited by 27
  10. 2025We Should Identify and Mitigate Third-Party Safety Risks in MCP-Powered Agent Systemscited by 27
  11. 2024To Trust or Not to Trust? Enhancing Large Language Models' Situated Faithfulness to External Contextscited by 25
  12. 2025FlippedRAG: Black-Box Opinion Manipulation Adversarial Attacks to Retrieval-Augmented Generation Modelscited by 24
  13. 2025HopRAG: Multi-Hop Reasoning for Logic-Aware Retrieval-Augmented Generationcited by 24
  14. 2025GraphRAG under Firecited by 23
  15. 2025Real AI Agents with Fake Memories: Fatal Context Manipulation Attacks on Web3 Agentscited by 23
  16. 2025RAG LLMs are Not Safer: A Safety Analysis of Retrieval-Augmented Generation for Large Language Modelscited by 23
  17. 2024CrAM: Credibility-Aware Attention Modification in LLMs for Combating Misinformation in RAGcited by 22
  18. 2024RAG-RewardBench: Benchmarking Reward Models in Retrieval Augmented Generation for Preference Alignmentcited by 22
  19. 2025U-NIAH: Unified RAG and LLM Evaluation for Long Context Needle-In-A-Haystackcited by 20
  20. 2024ATM: Adversarial Tuning Multi-agent System Makes a Robust Retrieval-Augmented Generatorcited by 19
  21. 2024Retrieval-Augmented Generation with Estimation of Source Reliabilitycited by 19
  22. 2025Secure Retrieval-Augmented Generation against Poisoning Attackscited by 19
  23. 2025Benchmarking Poisoning Attacks against Retrieval-Augmented Generationcited by 17
  24. 2025ReliabilityRAG: Effective and Provably Robust Defense for RAG-based Web-Searchcited by 17
  25. 2024Adversarial Decoding: Generating Readable Documents for Adversarial Objectivescited by 16
  26. 2024Towards More Robust Retrieval-Augmented Generation: Evaluating RAG Under Adversarial Poisoning Attackscited by 14
  27. 2025Think Twice Before You Act: Enhancing Agent Behavioral Safety with Thought Correctioncited by 14
  28. 2024Pandora's Box or Aladdin's Lamp: A Comprehensive Analysis Revealing the Role of RAG Noise in Large Language Modelscited by 12
  29. 2025Who Taught the Lie? Responsibility Attribution for Poisoned Knowledge in Retrieval-Augmented Generationcited by 12
  30. 2025SeCon-RAG: A Two-Stage Semantic Filtering and Conflict-Free Framework for Trustworthy RAGcited by 12
  31. 2024Black-Box Opinion Manipulation Attacks to Retrieval-Augmented Generation of Large Language Modelscited by 11
  32. 2025TracLLM: A Generic Framework for Attributing Long Context LLMscited by 11
  33. 2025Defending Against Knowledge Poisoning Attacks During Retrieval-Augmented Generationcited by 11
  34. 2026OpenDecoder: Open Large Language Model Decoding to Incorporate Document Quality in RAGcited by 11
  35. 2025A Survey of Scaling in Large Language Model Reasoningcited by 10
  36. 2025RoseRAG: Robust Retrieval-augmented Generation with Small-scale LLMs via Margin-aware Preference Optimizationcited by 9
  37. 2025Rescuing the Unpoisoned: Efficient Defense against Knowledge Corruption Attacks on RAG Systemscited by 9
  38. 2025Systems Security Foundations for Agentic Computingcited by 9

50 most cited of 144 All 144 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2026Retrieved But Not Reliable: A Survey on Attacks, and Defenses in Retrieval-Augmented Generation
  2. 2025Practical Poisoning Attacks against Retrieval-Augmented Generation
  3. 2026In RAG We Trust? Measuring Robustness of Retrieval-Augmented Generation Under Document Poisoning
  4. 2026Towards Secure Retrieval-Augmented Generation: A Comprehensive Review of Threats, Defenses and Benchmarks
  5. 2025RAGPart & RAGMask: Retrieval-Stage Defenses Against Corpus Poisoning in Retrieval-Augmented Generation
  6. 2026Detecting RAG Extraction Attack via Dual-Path Runtime Integrity Game
  7. 2024On the Vulnerability of Applying Retrieval-Augmented Generation within Knowledge-Intensive Application Domains
  8. 2026Beyond Explicit Refusals: Soft-Failure Attacks on Retrieval-Augmented Generation