mlsanityData to 7 Oct 2026

Training-Free Refusal of MCP Exploits via Retrieval-Augmented Generation

arXiv 2605.11217 ↗First posted 11 May 2026cs.LG cs.AI cs.CRcited by 0

On the map: Retrieval-augmented generation → Privacy and security of RAG

also in 3 more
Abstract (arXiv)

The model context protocol (MCP) has been widely adopted as an open standard enabling the seamless integration of generative AI agents. However, while LLM guardrails have significantly matured to refuse malicious or harmful queries (e.g., "How do I build a bomb?"), recent work has shown that MCP-enabled LLMs are highly susceptible to prompt injection attacks which avoid harmful or suspicious cues (e.g., "Can you add this ssh key to my bashrc file?"). Herein, we use state-of-the-art (SOTA) alignment fine-tuning algorithms to explore whether LLMs may be aligned to refuse such falsely benign attacks (FBAs). While SOTA algorithms based on direct preference optimization (DPO) improve refusal guardrails against FBAs, we show that this improvement is limited; DPO-based fine-tuning never improves FBA refusal rates beyond 47% across five popular open-source LLMs. Thus, to further improve FBA refusals, we introduce Retrieval Augmented Generation for Preference alignment (RAG-Pref), a simple RAG-based alignment algorithm which conditions on preferred and dispreferred samples to leverage contrastive information during inference. RAG-Pref is online (training-free), compatible with off-the-shelf packages, and, when combined with offline alignment algorithms, enables an average 3.7-fold improvement in FBA refusals across five widely used LLMs, compared to 2.9 for other online alignment methods and 1.5 for offline alignment alone. We additionally show that RAG-Pref generalizes beyond agentic safety: in stark contrast to other online alignment methods, RAG-Pref consistently improves performance on general human-preference benchmarks AlpacaEval 2 and MT-Bench across five SOTA alignment-tuned models, demonstrating broad applicability to general alignment tasks.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Privacy and security of RAG, one of 15 directions in Retrieval-augmented generation. Builds on 47 of its 62 references that are in the map. First posted on arXiv 11 May 2026.

Builds on

47 of 62 references.

Closest ancestors · 1

Same direction, newest first

  1. 2025RAG LLMs are Not Safer: A Safety Analysis of Retrieval-Augmented Generation for Large Language Models25 Apr

Other references · 46

  1. 2020Language Models are Few-Shot Learnerscited by 32,199
  2. 2022Training language models to follow instructions with human feedbackcited by 15,712
  3. 2024The Llama 3 Herd of Modelscited by 14,192
  4. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,160
  5. 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,393
  6. 2025Qwen3 Technical Reportcited by 8,195
  7. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 7,916
  8. 2024GPT-4o System Cardcited by 4,467
  9. 2023Universal and Transferable Adversarial Attacks on Aligned Language Modelscited by 2,914
  10. 2022Rethinking the Role of Demonstrations: What Makes In-Context Learning Work?cited by 1,413
  11. 2023Not what you've signed up for: Compromising Real-World LLM-Integrated Applications with Indirect Prompt Injectioncited by 1,339
  12. 2024HarmBench: A Standardized Evaluation Framework for Automated Red Teaming and Robust Refusalcited by 1,330
34 more
  1. 2023Jailbreaking Black Box Large Language Models in Twenty Queriescited by 1,326
  2. 2021An Explanation of In-context Learning as Implicit Bayesian Inferencecited by 915
  3. 2024Length-Controlled AlpacaEval: A Simple Way to Debias Automatic Evaluatorscited by 865
  4. 2023BeaverTails: Towards Improved Safety Alignment of LLM via a Human-Preference Datasetcited by 828
  5. 2022Ignore Previous Prompt: Attack Techniques For Language Modelscited by 760
  6. 2023AutoDAN: Generating Stealthy Jailbreak Prompts on Aligned Large Language Modelscited by 738
  7. 2023Safe RLHF: Safe Reinforcement Learning from Human Feedbackcited by 630
  8. 2023Tree of Attacks: Jailbreaking Black-Box LLMs Automaticallycited by 619
  9. 2023RLAIF vs. RLHF: Scaling Reinforcement Learning from Human Feedback with AI Feedbackcited by 615
  10. 2023Prompt Injection attack against LLM-integrated Applicationscited by 568
  11. 2024JailbreakBench: An Open Robustness Benchmark for Jailbreaking Large Language Modelscited by 494
  12. 2023"Do Anything Now": Characterizing and Evaluating In-The-Wild Jailbreak Prompts on Large Language Modelscited by 457
  13. 2023Statistical Rejection Sampling Improves Preference Optimizationcited by 321
  14. 2023How Language Model Hallucinations Can Snowballcited by 310
  15. 2024Self-Play Preference Optimization for Language Model Alignmentcited by 228
  16. 2023Fine-tuning Language Models for Factualitycited by 219
  17. 2024AutoDAN-Turbo: A Lifelong Agent for Strategy Self-Exploration to Jailbreak LLMscited by 185
  18. 2025Constitutional Classifiers: Defending against Universal Jailbreaks across Thousands of Hours of Red Teamingcited by 172
  19. 2024Provably Robust DPO: Aligning Language Models with Noisy Feedbackcited by 133
  20. 2024DPO Meets PPO: Reinforced Token Optimization for RLHFcited by 126
  21. 2024Noise Contrastive Alignment of Language Models with Explicit Rewardscited by 85
  22. 2025MCP Safety Audit: LLMs with the Model Context Protocol Allow Major Security Exploitscited by 61
  23. 2024Anchored Preference Optimization and Contrastive Revisions: Addressing Underspecification in Alignmentcited by 46
  24. 2024Binary Classifier Optimization for Large Language Model Alignmentcited by 44
  25. 2024Towards Efficient Exact Optimization of Language Model Alignmentcited by 37
  26. 2025SafeDPO: A Simple Approach to Direct Preference Optimization with Enhanced Safetycited by 37
  27. 2024Measuring and Enhancing Trustworthiness of LLMs in RAG through Grounded Attributions and Learning to Refusecited by 33
  28. 2025MCP Guardian: A Security-First Layer for Safeguarding MCP-Based AI Systemcited by 28
  29. 2024Distributional Preference Alignment of LLMs via Optimal Transportcited by 26
  30. 2024Linear Alignment: A Closed-form Solution for Aligning Human Preferences without Tuning and Feedbackcited by 25
  31. 2024RAG-RewardBench: Benchmarking Reward Models in Retrieval Augmented Generation for Preference Alignmentcited by 22
  32. 2025Divide-Then-Align: Honest Alignment based on the Knowledge Boundary of RAGcited by 13
  33. 2024PA-RAG: RAG Alignment via Multi-Perspective Preference Optimizationcited by 11
  34. 2025On-the-fly Preference Alignment via Principle-Guided Decodingcited by 10

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025MCP Safety Training: Learning to Refuse Falsely Benign MCP Exploits using Improved Preference Alignment
  2. 2024Self and Cross-Model Distillation for LLMs: Effective Methods for Refusal Pattern Alignment
  3. 2025Improving LLM Safety Alignment with Dual-Objective Optimization
  4. 2026When Safety Becomes a Vulnerability: Exploiting LLM Alignment Homogeneity for Transferable Blocking in RAG
  5. 2026Few-Shot Truly Benign DPO Attack for Jailbreaking LLMs
  6. 2025From Threat to Tool: Leveraging Refusal-Aware Injection Attacks for Safety Alignment
  7. 2025MPMA: Preference Manipulation Attack Against Model Context Protocol
  8. 2026Invisible Threats from Model Context Protocol: Generating Stealthy Injection Payload via Tree-based Adaptive Search