mlsanityData to 7 Oct 2026

RAG LLMs are Not Safer: A Safety Analysis of Retrieval-Augmented Generation for Large Language Models

arXiv 2504.18041 ↗First posted 25 Apr 2025cs.CL cs.AIcited by 23

On the map: Retrieval-augmented generation → Privacy and security of RAG

also in 1 more
Abstract (arXiv)

Efforts to ensure the safety of large language models (LLMs) include safety fine-tuning, evaluation, and red teaming. However, despite the widespread use of the Retrieval-Augmented Generation (RAG) framework, AI safety work focuses on standard LLMs, which means we know little about how RAG use cases change a model's safety profile. We conduct a detailed comparative analysis of RAG and non-RAG frameworks with eleven LLMs. We find that RAG can make models less safe and change their safety profile. We explore the causes of this change and find that even combinations of safe models with safe documents can cause unsafe generations. In addition, we evaluate some existing red teaming methods for RAG settings and show that they are less effective than when used for non-RAG settings. Our work highlights the need for safety research and red-teaming methods specifically tailored for RAG LLMs.

Key facts. Cited by 23 papers in the map (data through 7 Oct 2026). Placed in Privacy and security of RAG, one of 15 directions in Retrieval-augmented generation. Builds on 64 of its 97 references that are in the map. First posted on arXiv 25 Apr 2025.

Builds on

64 of 97 references.

Closest ancestors · 5

Same direction, newest first

  1. 2024Machine Against the RAG: Jamming Retrieval-Augmented Generation with Blocker Documents9 Jun
  2. 2024Certifiably Robust RAG against Retrieval Corruption24 May
  3. 2024Typos that Broke the RAG's Back: Genetic Attack on RAG Pipeline by Simulating Documents in the Wild via Low-level Perturbations22 Apr
  4. 2024The Good and The Bad: Exploring Privacy Issues in Retrieval-Augmented Generation (RAG)23 Feb
  5. 2024Pandora: Jailbreak GPTs by Retrieval Augmented Generation Poisoning13 Feb

Other references · 59

  1. 2022Training language models to follow instructions with human feedbackcited by 15,712
  2. 2024The Llama 3 Herd of Modelscited by 14,192
  3. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,865
  4. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,160
  5. 2020Dense Passage Retrieval for Open-Domain Question Answeringcited by 3,779
  6. 2022Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedbackcited by 3,634
  7. 2023Universal and Transferable Adversarial Attacks on Aligned Language Modelscited by 2,914
  8. 2023Code Llama: Open Foundation Models for Codecited by 2,105
  9. 2024Phi-3 Technical Report: A Highly Capable Language Model Locally on Your Phonecited by 1,883
  10. 2023ToolLLM: Facilitating Large Language Models to Master 16000+ Real-world APIscited by 1,880
  11. 2023Jailbroken: How Does LLM Safety Training Fail?cited by 1,565
  12. 2023Not what you've signed up for: Compromising Real-World LLM-Integrated Applications with Indirect Prompt Injectioncited by 1,339
38 more
  1. 2024HarmBench: A Standardized Evaluation Framework for Automated Red Teaming and Robust Refusalcited by 1,330
  2. 2023Jailbreaking Black Box Large Language Models in Twenty Queriescited by 1,326
  3. 2023Llama Guard: LLM-based Input-Output Safeguard for Human-AI Conversationscited by 1,124
  4. 2022Red Teaming Language Models with Language Modelscited by 983
  5. 2023BloombergGPT: A Large Language Model for Financecited by 923
  6. 2023BeaverTails: Towards Improved Safety Alignment of LLM via a Human-Preference Datasetcited by 828
  7. 2024Medusa: Simple LLM Inference Acceleration Framework with Multiple Decoding Headscited by 821
  8. 2023OpenAssistant Conversations -- Democratizing Large Language Model Alignmentcited by 712
  9. 2021Retrieval Augmentation Reduces Hallucination in Conversationcited by 691
  10. 2023Baseline Defenses for Adversarial Attacks Against Aligned Language Modelscited by 608
  11. 2024How Johnny Can Persuade LLMs to Jailbreak Them: Rethinking Persuasion to Challenge AI Safety by Humanizing LLMscited by 533
  12. 2023Jailbreaking ChatGPT via Prompt Engineering: An Empirical Studycited by 478
  13. 2024Jailbreaking Leading Safety-Aligned LLMs with Simple Adaptive Attackscited by 450
  14. 2024AgentPoison: Red-teaming LLM Agents via Poisoning Memory or Knowledge Basescited by 402
  15. 2023Large Language Models as Tool Makerscited by 293
  16. 2024R-Judge: Benchmarking Safety Risk Awareness for LLM Agentscited by 268
  17. 2024SafeDecoding: Defending against Jailbreak Attacks via Safety-Aware Decodingcited by 233
  18. 2024ShieldGemma: Generative AI Content Moderation Based on Gemmacited by 224
  19. 2023Red-Teaming Large Language Models using Chain of Utterances for Safety-Alignmentcited by 213
  20. 2024COLD-Attack: Jailbreaking LLMs with Stealthiness and Controllabilitycited by 162
  21. 2023MART: Improving LLM Safety with Multi-round Automatic Red-Teamingcited by 161
  22. 2024Latent Adversarial Training Improves Robustness to Persistent Harmful Behaviors in LLMscited by 158
  23. 2023Poisoning Retrieval Corpora by Injecting Adversarial Passagescited by 153
  24. 2024The Emerged Security and Privacy of LLM Agent: A Survey with Case Studiescited by 127
  25. 2023DISC-LawLLM: Fine-tuning Large Language Models for Intelligent Legal Servicescited by 117
  26. 2024GuardAgent: Safeguard LLM Agents by a Guard Agent via Knowledge-Enabled Reasoningcited by 113
  27. 2023Explore, Establish, Exploit: Red Teaming Language Models from Scratchcited by 111
  28. 2024ClashEval: Quantifying the tug-of-war between an LLM's internal prior and external evidencecited by 109
  29. 2023Evil Geniuses: Delving into the Safety of LLM-based Agentscited by 103
  30. 2024ALERT: A Comprehensive Benchmark for Assessing Large Language Models' Safety through Red Teamingcited by 96
  31. 2024How Alignment and Jailbreak Work: Explain LLM Safety through Intermediate Hidden Statescited by 92
  32. 2024Trustworthiness in Retrieval-Augmented Generation Systems: A Surveycited by 83
  33. 2024Red-Teaming for Generative AI: Silver Bullet or Security Theater?cited by 81
  34. 2024Reliable, Adaptable, and Attributable Language Models with Retrievalcited by 73
  35. 2024Improved Few-Shot Jailbreaking Can Circumvent Aligned Language Models and Their Defensescited by 68
  36. 2024JailbreakZoo: Survey, Landscapes, and Horizons in Jailbreaking Large Language and Vision-Language Modelscited by 67
  37. 2024Retrieval Augmented Generation (RAG) and Beyond: A Comprehensive Survey on How to Make your LLMs use External Data More Wiselycited by 65
  38. 2024Introducing v0.5 of the AI Safety Benchmark from MLCommonscited by 60

50 most cited of 59 All 59 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 17; 2026: 6.

23 papers.

Sort:

By area:

  1. 2025A Survey of LLM-Driven AI Agent Communication: Protocols, Security Risks, and Defense Countermeasurescited by 54
  2. 2025SafeSearch: Do Not Trade Safety for Utility in LLM Search Agentscited by 9
  3. 2025Scalable Defense against In-the-wild Jailbreaking Attacks with Safety Context Retrievalcited by 7
  4. 2025NeuroGenPoisoning: Neuron-Guided Attacks on Retrieval-Augmented Generation of LLM via Genetic Optimization of External Knowledgecited by 7
  5. 2025RAR: Setting Knowledge Tripwires for Retrieval Augmented Rejectioncited by 3
  6. 2025MCP Safety Training: Learning to Refuse Falsely Benign MCP Exploits using Improved Preference Alignmentcited by 3
  7. 2025RAG Makes Guardrails Unsafe? Investigating Robustness of Guardrails under RAG-style Contextscited by 3
  8. 2025Safety Degradation in AI Agentscited by 2
  9. 2025Fine-Grained Privacy Extraction from Retrieval-Augmented Generation Systems via Knowledge Asymmetry Exploitationcited by 2
  10. 2025Context-Aware Pragmatic Metacognitive Prompting for Sarcasm Detectioncited by 2
  11. 2025A Large-Language-Model Framework for Automated Humanitarian Situation Reportingcited by 2
  12. 2025AC-LoRA: (Almost) Training-Free Access Control-Aware Multi-Modal LLMscited by 1
11 more
  1. 2025When Search Goes Wrong: Red-Teaming Web-Augmented Large Language Modelscited by 1
  2. 2025Data Security in Large Language Models: Risks, Defense, and Directionscited by 0
  3. 2025Adversarial Distilled Retrieval-Augmented Guarding Model for Online Malicious Intent Detectioncited by 0
  4. 2025On the Sensitivity of Instruction-tuned LLMs to Harmful Sentences in Long Inputscited by 0
  5. 2025Quantifying Document Impact in RAG-LLMscited by 0
  6. 2026The Cost of Context: Mitigating Textual Bias in Multimodal Retrieval-Augmented Generationcited by 0
  7. 2026Training-Free Refusal of MCP Exploits via Retrieval-Augmented Generationcited by 0
  8. 2026Safety Measurements for Fine-tuned LLMs Should be Grounded in Capabilitycited by 0
  9. 2026Security and Privacy in Retrieval-Augmented Generation: Architectures, Threats, Defenses, and Future Directions for Building Trustworthy Systemscited by 0
  10. 2026The Tokens Remember: When Tokenization Bypasses Knowledge Editing and Unlearningcited by 0
  11. 2026Mapping the RAG Landscape: A Four Axis Taxonomy of Efficiency, Defense, Interactivity, and Reasoningcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026RAG-Safety-Bench: Reliable Evaluation of Retrieval-Augmented LLM Safety
  2. 2024C-RAG: Certified Generation Risks for Retrieval-Augmented Language Models
  3. 2025Evaluating the Retrieval Robustness of Large Language Models
  4. 2025Towards Trustworthy Retrieval Augmented Generation for Large Language Models: A Survey
  5. 2024Large Language Model Safety: A Holistic Survey
  6. 2023Retrieval-Augmented Generation for Large Language Models: A Survey
  7. 2026SoK: Privacy Risks and Mitigations in Retrieval-Augmented Generation Systems
  8. 2023All Languages Matter: On the Multilingual Safety of Large Language Models