mlsanityData to 7 Oct 2026

SafeRAG: Benchmarking Security in Retrieval-Augmented Generation of Large Language Model

arXiv 2501.18636 ↗First posted 28 Jan 2025cs.CR cs.AI cs.IRcited by 31

On the map: Retrieval-augmented generation → Privacy and security of RAG

Abstract (arXiv)

The indexing-retrieval-generation paradigm of retrieval-augmented generation (RAG) has been highly successful in solving knowledge-intensive tasks by integrating external knowledge into large language models (LLMs). However, the incorporation of external and unverified knowledge increases the vulnerability of LLMs because attackers can perform attack tasks by manipulating knowledge. In this paper, we introduce a benchmark named SafeRAG designed to evaluate the RAG security. First, we classify attack tasks into silver noise, inter-context conflict, soft ad, and white Denial-of-Service. Next, we construct RAG security evaluation dataset (i.e., SafeRAG dataset) primarily manually for each task. We then utilize the SafeRAG dataset to simulate various attack scenarios that RAG may encounter. Experiments conducted on 14 representative RAG components demonstrate that RAG exhibits significant vulnerability to all attack tasks and even the most apparent attack task can easily bypass existing retrievers, filters, or advanced LLMs, resulting in the degradation of RAG service quality. Code is available at: https://github.com/IAAR-Shanghai/SafeRAG.

Key facts. Cited by 31 papers in the map (data through 7 Oct 2026). Placed in Privacy and security of RAG, one of 15 directions in Retrieval-augmented generation. Builds on 15 of its 47 references that are in the map. First posted on arXiv 28 Jan 2025.

Builds on

15 of 47 references.

Closest ancestors · 2

Same direction, newest first

  1. 2024Machine Against the RAG: Jamming Retrieval-Augmented Generation with Blocker Documents9 Jun
  2. 2024Certifiably Robust RAG against Retrieval Corruption24 May

Other references · 13

  1. 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,049
  2. 2022Ignore Previous Prompt: Attack Techniques For Language Modelscited by 760
  3. 2023Toxicity in ChatGPT: Analyzing Persona-assigned Language Modelscited by 356
  4. 2024Retrieval-Augmented Generation for AI-Generated Content: A Surveycited by 352
  5. 2024Knowledge Conflicts for LLMs: A Surveycited by 274
  6. 2023Poisoning Retrieval Corpora by Injecting Adversarial Passagescited by 153
  7. 2024ClashEval: Quantifying the tug-of-war between an LLM's internal prior and external evidencecited by 109
  8. 2024Enhancing Noise Robustness of Retrieval-Augmented Language Models with Adaptive Adversarial Trainingcited by 88
  9. 2023RECALL: A Benchmark for LLMs Robustness against External Counterfactual Knowledgecited by 51
  10. 2022Synthetic Disinformation Attacks on Automated Fact Verification Systemscited by 44
  11. 2021Attacking Open-domain Question Answering by Injecting Misinformationcited by 41
  12. 2024Small Models, Big Insights: Leveraging Slim Proxy Models To Decide When and What to Retrieve for LLMscited by 34
1 more
  1. 2024Supportiveness-based Knowledge Rewriting for Retrieval-augmented Language Modelingcited by 4

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 13; 2026: 18.

31 papers.

Sort:

By area:

  1. 2025A Survey on Knowledge-Oriented Retrieval-Augmented Generationcited by 34
  2. 2025A Systematic Review of Key Retrieval-Augmented Generation (RAG) Systems: Progress, Gaps, and Future Directionscited by 30
  3. 2025HopRAG: Multi-Hop Reasoning for Logic-Aware Retrieval-Augmented Generationcited by 24
  4. 2025Retrieval Augmented Generation Evaluation in the Era of Large Language Models: A Comprehensive Surveycited by 20
  5. 2025Benchmarking Poisoning Attacks against Retrieval-Augmented Generationcited by 17
  6. 2025Bias Amplification in RAG: Poisoning Knowledge Retrieval to Steer LLMscited by 8
  7. 2026ToolPrivacyBench: Benchmarking Purpose-Bound Privacy in Tool-Using LLM Agentscited by 6
  8. 2025RAGuard: A Novel Approach for in-context Safe Retrieval Augmented Generation for LLMscited by 5
  9. 2025SafeSearch: Automated Red-Teaming of LLM-Based Search Agentscited by 4
  10. 2026When Safety Becomes a Vulnerability: Exploiting LLM Alignment Homogeneity for Transferable Blocking in RAGcited by 4
  11. 2025RAR: Setting Knowledge Tripwires for Retrieval Augmented Rejectioncited by 3
  12. 2025RAG Makes Guardrails Unsafe? Investigating Robustness of Guardrails under RAG-style Contextscited by 3
19 more
  1. 2025Steering Over-refusals Towards Safety in Retrieval Augmented Generationcited by 1
  2. 2025Bilevel Optimization for Covert Memory Tampering in Heterogeneous Multi-Agent Architectures (XAMT)cited by 1
  3. 2026Retrieval Pivot Attacks in Hybrid RAG: Measuring and Mitigating Amplified Leakage from Vector Seeds to Graph Expansioncited by 1
  4. 2026LLM-Oriented Information Retrieval: A Denoising-First Perspectivecited by 1
  5. 2026Towards Dependable Retrieval-Augmented Generation Using Factual Confidence Predictioncited by 1
  6. 2026Taxonomy and Consistency Analysis of Safety Benchmarks for AI Agentscited by 1
  7. 2026A Failure-Mode Benchmark for Polymorphic Sybil Poisoning in RAGcited by 1
  8. 2026When Context Bites: Detecting RAG Poisoning via Document-Level Attention Collapsecited by 1
  9. 2025Harmonia: End-to-End RAG Serving Optimizationcited by 0
  10. 2026Beyond Explicit Refusals: Soft-Failure Attacks on Retrieval-Augmented Generationcited by 0
  11. 2026Adaptive Defense Orchestration for RAG: A Sentinel-Strategist Architecture against Multi-Vector Attackscited by 0
  12. 2026The Periodic Table of LLM Reasoning: A Structured Survey of Reasoning Paradigms, Methods, and Failure Modescited by 0
  13. 2026Influence Factors on RAG Poisoningcited by 0
  14. 2026Security and Privacy in Retrieval-Augmented Generation: Architectures, Threats, Defenses, and Future Directions for Building Trustworthy Systemscited by 0
  15. 2026Tracing Target Answers in Poisoned Retrieval Corpora via Token Influence Attributioncited by 0
  16. 2026Defending Retrieval-Augmented Intrusion Detection Against Knowledge Poisoning and Prompt Injectioncited by 0
  17. 2026Context Inference Attacks Without Jailbreakscited by 0
  18. 2026When Tools Silently Lie: Evaluating and Mitigating Blind Compliance in Tool-Augmented Data Agentscited by 0
  19. 2026Mapping the RAG Landscape: A Four Axis Taxonomy of Efficiency, Defense, Interactivity, and Reasoningcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026Retrieved But Not Reliable: A Survey on Attacks, and Defenses in Retrieval-Augmented Generation
  2. 2025Token-Level Precise Attack on RAG: Searching for the Best Alternatives to Mislead Generation
  3. 2026Securing Retrieval-Augmented Generation: A Taxonomy of Attacks, Defenses, and Future Directions
  4. 2024PoisonedRAG: Knowledge Corruption Attacks to Retrieval-Augmented Generation of Large Language Models
  5. 2026Benchmarking Knowledge-Extraction Attack and Defense on Retrieval-Augmented Generation
  6. 2024Data Extraction Attacks in Retrieval-Augmented Generation via Backdoors
  7. 2026Towards Secure Retrieval-Augmented Generation: A Comprehensive Review of Threats, Defenses and Benchmarks
  8. 2025RAG Safety: Exploring Knowledge Poisoning Attacks to Retrieval-Augmented Generation