# SafeRAG: Benchmarking Security in Retrieval-Augmented Generation of Large Language Model

arXiv: https://arxiv.org/abs/2501.18636 · HTML: https://mlsanity.com/paper/2501.18636/

## Key facts

- Cited by 32 papers in the map (data through 14 Oct 2026).
- Placed in Privacy and security of RAG, one of 15 directions in Retrieval-augmented generation.
- Builds on 15 of its 47 references that are in the map.
- First posted on arXiv 28 Jan 2025.

## Placed in

- Retrieval-augmented generation / [Privacy and security of RAG](https://mlsanity.com/rag/privacy-and-security-of-rag/)

## Builds on (most cited first, 15 of 15)

- [Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks](https://mlsanity.com/paper/1908.10084/) (2019)
- [Ignore Previous Prompt: Attack Techniques For Language Models](https://mlsanity.com/paper/2211.09527/) (2022)
- [Toxicity in ChatGPT: Analyzing Persona-assigned Language Models](https://mlsanity.com/paper/2304.05335/) (2023)
- [Retrieval-Augmented Generation for AI-Generated Content: A Survey](https://mlsanity.com/paper/2402.19473/) (2024)
- [Knowledge Conflicts for LLMs: A Survey](https://mlsanity.com/paper/2403.08319/) (2024)
- [Poisoning Retrieval Corpora by Injecting Adversarial Passages](https://mlsanity.com/paper/2310.19156/) (2023)
- [Certifiably Robust RAG against Retrieval Corruption](https://mlsanity.com/paper/2405.15556/) (2024)
- [ClashEval: Quantifying the tug-of-war between an LLM's internal prior and external evidence](https://mlsanity.com/paper/2404.10198/) (2024)
- [Machine Against the RAG: Jamming Retrieval-Augmented Generation with Blocker Documents](https://mlsanity.com/paper/2406.05870/) (2024)
- [Enhancing Noise Robustness of Retrieval-Augmented Language Models with Adaptive Adversarial Training](https://mlsanity.com/paper/2405.20978/) (2024)
- [RECALL: A Benchmark for LLMs Robustness against External Counterfactual Knowledge](https://mlsanity.com/paper/2311.08147/) (2023)
- [Synthetic Disinformation Attacks on Automated Fact Verification Systems](https://mlsanity.com/paper/2202.09381/) (2022)
- [Attacking Open-domain Question Answering by Injecting Misinformation](https://mlsanity.com/paper/2110.07803/) (2021)
- [Small Models, Big Insights: Leveraging Slim Proxy Models To Decide When and What to Retrieve for LLMs](https://mlsanity.com/paper/2402.12052/) (2024)
- [Supportiveness-based Knowledge Rewriting for Retrieval-augmented Language Modeling](https://mlsanity.com/paper/2406.08116/) (2024)

## Built on by (most cited first, 30 of 32)

- [A Survey on Knowledge-Oriented Retrieval-Augmented Generation](https://mlsanity.com/paper/2503.10677/) (2025)
- [A Systematic Review of Key Retrieval-Augmented Generation (RAG) Systems: Progress, Gaps, and Future Directions](https://mlsanity.com/paper/2507.18910/) (2025)
- [HopRAG: Multi-Hop Reasoning for Logic-Aware Retrieval-Augmented Generation](https://mlsanity.com/paper/2502.12442/) (2025)
- [Retrieval Augmented Generation Evaluation in the Era of Large Language Models: A Comprehensive Survey](https://mlsanity.com/paper/2504.14891/) (2025)
- [Benchmarking Poisoning Attacks against Retrieval-Augmented Generation](https://mlsanity.com/paper/2505.18543/) (2025)
- [Bias Amplification in RAG: Poisoning Knowledge Retrieval to Steer LLMs](https://mlsanity.com/paper/2506.11415/) (2025)
- [ToolPrivacyBench: Benchmarking Purpose-Bound Privacy in Tool-Using LLM Agents](https://mlsanity.com/paper/2606.28061/) (2026)
- [RAGuard: A Novel Approach for in-context Safe Retrieval Augmented Generation for LLMs](https://mlsanity.com/paper/2509.03768/) (2025)
- [The Automation Advantage in AI Red Teaming](https://mlsanity.com/paper/2504.19855/) (2025)
- [SafeSearch: Automated Red-Teaming of LLM-Based Search Agents](https://mlsanity.com/paper/2509.23694/) (2025)
- [When Safety Becomes a Vulnerability: Exploiting LLM Alignment Homogeneity for Transferable Blocking in RAG](https://mlsanity.com/paper/2603.03919/) (2026)
- [Taxonomy and Consistency Analysis of Safety Benchmarks for AI Agents](https://mlsanity.com/paper/2605.16282/) (2026)
- [RAR: Setting Knowledge Tripwires for Retrieval Augmented Rejection](https://mlsanity.com/paper/2505.13581/) (2025)
- [RAG Makes Guardrails Unsafe? Investigating Robustness of Guardrails under RAG-style Contexts](https://mlsanity.com/paper/2510.05310/) (2025)
- [Steering Over-refusals Towards Safety in Retrieval Augmented Generation](https://mlsanity.com/paper/2510.10452/) (2025)
- [Bilevel Optimization for Covert Memory Tampering in Heterogeneous Multi-Agent Architectures (XAMT)](https://mlsanity.com/paper/2512.15790/) (2025)
- [Retrieval Pivot Attacks in Hybrid RAG: Measuring and Mitigating Amplified Leakage from Vector Seeds to Graph Expansion](https://mlsanity.com/paper/2602.08668/) (2026)
- [LLM-Oriented Information Retrieval: A Denoising-First Perspective](https://mlsanity.com/paper/2605.00505/) (2026)
- [Towards Dependable Retrieval-Augmented Generation Using Factual Confidence Prediction](https://mlsanity.com/paper/2605.05244/) (2026)
- [A Failure-Mode Benchmark for Polymorphic Sybil Poisoning in RAG](https://mlsanity.com/paper/2607.03739/) (2026)
- [When Context Bites: Detecting RAG Poisoning via Document-Level Attention Collapse](https://mlsanity.com/paper/2608.06947/) (2026)
- [Harmonia: End-to-End RAG Serving Optimization](https://mlsanity.com/paper/2505.07833/) (2025)
- [Beyond Explicit Refusals: Soft-Failure Attacks on Retrieval-Augmented Generation](https://mlsanity.com/paper/2604.18663/) (2026)
- [Adaptive Defense Orchestration for RAG: A Sentinel-Strategist Architecture against Multi-Vector Attacks](https://mlsanity.com/paper/2604.20932/) (2026)
- [The Periodic Table of LLM Reasoning: A Structured Survey of Reasoning Paradigms, Methods, and Failure Modes](https://mlsanity.com/paper/2606.11470/) (2026)
- [Influence Factors on RAG Poisoning](https://mlsanity.com/paper/2606.12469/) (2026)
- [Security and Privacy in Retrieval-Augmented Generation: Architectures, Threats, Defenses, and Future Directions for Building Trustworthy Systems](https://mlsanity.com/paper/2606.25533/) (2026)
- [Tracing Target Answers in Poisoned Retrieval Corpora via Token Influence Attribution](https://mlsanity.com/paper/2606.25721/) (2026)
- [Defending Retrieval-Augmented Intrusion Detection Against Knowledge Poisoning and Prompt Injection](https://mlsanity.com/paper/2608.08100/) (2026)
- [Context Inference Attacks Without Jailbreaks](https://mlsanity.com/paper/2609.01663/) (2026)

## Abstract (arXiv)

The indexing-retrieval-generation paradigm of retrieval-augmented generation (RAG) has been highly successful in solving knowledge-intensive tasks by integrating external knowledge into large language models (LLMs). However, the incorporation of external and unverified knowledge increases the vulnerability of LLMs because attackers can perform attack tasks by manipulating knowledge. In this paper, we introduce a benchmark named SafeRAG designed to evaluate the RAG security. First, we classify attack tasks into silver noise, inter-context conflict, soft ad, and white Denial-of-Service. Next, we construct RAG security evaluation dataset (i.e., SafeRAG dataset) primarily manually for each task. We then utilize the SafeRAG dataset to simulate various attack scenarios that RAG may encounter. Experiments conducted on 14 representative RAG components demonstrate that RAG exhibits significant vulnerability to all attack tasks and even the most apparent attack task can easily bypass existing retrievers, filters, or advanced LLMs, resulting in the degradation of RAG service quality. Code is available at: https://github.com/IAAR-Shanghai/SafeRAG.