# DeRAG: Black-box Adversarial Attacks on Multiple Retrieval-Augmented Generation Applications via Prompt Injection

arXiv: https://arxiv.org/abs/2507.15042 · HTML: https://mlsanity.com/paper/2507.15042/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Placed in Privacy and security of RAG, one of 15 directions in Retrieval-augmented generation.
- Builds on 23 of its 34 references that are in the map.
- First posted on arXiv 20 Jul 2025.

## Placed in

- Retrieval-augmented generation / [Privacy and security of RAG](https://mlsanity.com/rag/privacy-and-security-of-rag/)
- LLM safety and security / [Membership inference attacks](https://mlsanity.com/safety/membership-inference-attacks/)

## Builds on (most cited first, 23 of 23)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [SQuAD: 100,000+ Questions for Machine Comprehension of Text](https://mlsanity.com/paper/1606.05250/) (2016)
- [Dense Passage Retrieval for Open-Domain Question Answering](https://mlsanity.com/paper/2004.04906/) (2020)
- [FEVER: a large-scale dataset for Fact Extraction and VERification](https://mlsanity.com/paper/1803.05355/) (2018)
- [BEIR: A Heterogenous Benchmark for Zero-shot Evaluation of Information Retrieval Models](https://mlsanity.com/paper/2104.08663/) (2021)
- [Ignore Previous Prompt: Attack Techniques For Language Models](https://mlsanity.com/paper/2211.09527/) (2022)
- [One pixel attack for fooling deep neural networks](https://mlsanity.com/paper/1710.08864/) (2017)
- [Fact or Fiction: Verifying Scientific Claims](https://mlsanity.com/paper/2004.14974/) (2020)
- [Release Strategies and the Social Impacts of Language Models](https://mlsanity.com/paper/1908.09203/) (2019)
- [Detecting Language Model Attacks with Perplexity](https://mlsanity.com/paper/2308.14132/) (2023)
- [Generative Representational Instruction Tuning](https://mlsanity.com/paper/2402.09906/) (2024)
- [EvoPrompt: Connecting LLMs with Evolutionary Algorithms Yields Powerful Prompt Optimizers](https://mlsanity.com/paper/2309.08532/) (2023)
- [Automatic and Universal Prompt Injection Attacks against Large Language Models](https://mlsanity.com/paper/2403.04957/) (2024)
- [Enhancing LLM Factual Accuracy with RAG to Counter Hallucinations: A Case Study on Domain-Specific Queries in Private Knowledge-Bases](https://mlsanity.com/paper/2403.10446/) (2024)
- [Black-box Prompt Learning for Pre-trained Language Models](https://mlsanity.com/paper/2201.08531/) (2022)
- [Arctic-Embed: Scalable, Efficient, and Accurate Text Embedding Models](https://mlsanity.com/paper/2405.05374/) (2024)
- [Harnessing the Universal Geometry of Embeddings](https://mlsanity.com/paper/2505.12540/) (2025)
- [PRADA: Practical Black-Box Adversarial Attacks against Neural Ranking Models](https://mlsanity.com/paper/2204.01321/) (2022)
- [Token-Level Adversarial Prompt Detection Based on Perplexity Measures and Contextual Information](https://mlsanity.com/paper/2311.11509/) (2023)
- [Goal-guided Generative Prompt Injection Attack on Large Language Models](https://mlsanity.com/paper/2404.07234/) (2024)
- [Prompt Perturbation in Retrieval-Augmented Generation based Large Language Models](https://mlsanity.com/paper/2402.07179/) (2024)
- [Targeting the Core: A Simple and Effective Method to Attack RAG-based Agents via Direct LLM Manipulation](https://mlsanity.com/paper/2412.04415/) (2024)
- [CtrlRAG: Black-box Document Poisoning Attacks for Retrieval-Augmented Generation of Large Language Models](https://mlsanity.com/paper/2503.06950/) (2025)

## Built on by (most cited first, 5 of 5)

- [Evaluating the Robustness of Retrieval-Augmented Generation to Adversarial Evidence in the Health Domain](https://mlsanity.com/paper/2509.03787/) (2025)
- [ER-MIA: Black-Box Adversarial Memory Injection Attacks on Long-Term Memory-Augmented Large Language Models](https://mlsanity.com/paper/2602.15344/) (2026)
- [RADAR: Defending RAG Dynamically against Retrieval Corruption](https://mlsanity.com/paper/2605.22041/) (2026)
- [Conflict-Aware Retriever Editing for Knowledge Injection Attacks on LLM-Based RAG Systems](https://mlsanity.com/paper/2606.18310/) (2026)
- [Security and Privacy in Retrieval-Augmented Generation: Architectures, Threats, Defenses, and Future Directions for Building Trustworthy Systems](https://mlsanity.com/paper/2606.25533/) (2026)

## Abstract (arXiv)

Adversarial prompt attacks can significantly alter the reliability of Retrieval-Augmented Generation (RAG) systems by re-ranking them to produce incorrect outputs. In this paper, we present a novel method that applies Differential Evolution (DE) to optimize adversarial prompt suffixes for RAG-based question answering. Our approach is gradient-free, treating the RAG pipeline as a black box and evolving a population of candidate suffixes to maximize the retrieval rank of a targeted incorrect document to be closer to real world scenarios. We conducted experiments on the BEIR QA datasets to evaluate attack success at certain retrieval rank thresholds under multiple retrieving applications. Our results demonstrate that DE-based prompt optimization attains competitive (and in some cases higher) success rates compared to GGPP to dense retrievers and PRADA to sparse retrievers, while using only a small number of tokens (<=5 tokens) in the adversarial suffix. Furthermore, we introduce a readability-aware suffix construction strategy, validated by a statistically significant reduction in MLM negative log-likelihood with Welch's t-test. Through evaluations with a BERT-based adversarial suffix detector, we show that DE-generated suffixes evade detection, yielding near-chance detection accuracy.