# The RAG Paradox: A Black-Box Attack Exploiting Unintentional Vulnerabilities in Retrieval-Augmented Generation Systems

arXiv: https://arxiv.org/abs/2502.20995 · HTML: https://mlsanity.com/paper/2502.20995/

## Key facts

- Cited by 14 papers in the map (data through 14 Oct 2026).
- Placed in Privacy and security of RAG, one of 15 directions in Retrieval-augmented generation.
- Builds on 28 of its 39 references that are in the map.
- First posted on arXiv 28 Feb 2025.

## Placed in

- Retrieval-augmented generation / [Privacy and security of RAG](https://mlsanity.com/rag/privacy-and-security-of-rag/)

## Builds on (most cited first, 28 of 28)

- [The Llama 3 Herd of Models](https://mlsanity.com/paper/2407.21783/) (2024)
- [Llama 2: Open Foundation and Fine-Tuned Chat Models](https://mlsanity.com/paper/2307.09288/) (2023)
- [Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks](https://mlsanity.com/paper/2005.11401/) (2020)
- [SQuAD: 100,000+ Questions for Machine Comprehension of Text](https://mlsanity.com/paper/1606.05250/) (2016)
- [HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answering](https://mlsanity.com/paper/1809.09600/) (2018)
- [What Disease does this Patient Have? A Large-scale Open Domain Question Answering Dataset from Medical Exams](https://mlsanity.com/paper/2009.13081/) (2020)
- [BEIR: A Heterogenous Benchmark for Zero-shot Evaluation of Information Retrieval Models](https://mlsanity.com/paper/2104.08663/) (2021)
- [Unsupervised Dense Information Retrieval with Contrastive Learning](https://mlsanity.com/paper/2112.09118/) (2021)
- [Leveraging Passage Retrieval with Generative Models for Open Domain Question Answering](https://mlsanity.com/paper/2007.01282/) (2020)
- [Interleaving Retrieval with Chain-of-Thought Reasoning for Knowledge-Intensive Multi-Step Questions](https://mlsanity.com/paper/2212.10509/) (2022)
- [Approximate Nearest Neighbor Negative Contrastive Learning for Dense Text Retrieval](https://mlsanity.com/paper/2007.00808/) (2020)
- [C-Pack: Packed Resources For General Chinese Embeddings](https://mlsanity.com/paper/2309.07597/) (2023)
- [AgentPoison: Red-teaming LLM Agents via Poisoning Memory or Knowledge Bases](https://mlsanity.com/paper/2407.12784/) (2024)
- [Benchmarking Retrieval-Augmented Generation for Medicine](https://mlsanity.com/paper/2402.13178/) (2024)
- [Text Embeddings Reveal (Almost) As Much As Text](https://mlsanity.com/paper/2310.06816/) (2023)
- [RankT5: Fine-Tuning T5 for Text Ranking with Ranking Losses](https://mlsanity.com/paper/2210.10634/) (2022)
- [Machine Against the RAG: Jamming Retrieval-Augmented Generation with Blocker Documents](https://mlsanity.com/paper/2406.05870/) (2024)
- [BM25S: Orders of magnitude faster lexical search via eager sparse scoring](https://mlsanity.com/paper/2407.03618/) (2024)
- [Typos that Broke the RAG's Back: Genetic Attack on RAG Pipeline by Simulating Documents in the Wild via Low-level Perturbations](https://mlsanity.com/paper/2404.13948/) (2024)
- ["Glue pizza and eat rocks" -- Exploiting Vulnerabilities in Retrieval-Augmented Generative Models](https://mlsanity.com/paper/2406.19417/) (2024)
- [Attacking Open-domain Question Answering by Injecting Misinformation](https://mlsanity.com/paper/2110.07803/) (2021)
- [Neural Retrievers are Biased Towards LLM-Generated Content](https://mlsanity.com/paper/2310.20501/) (2023)
- [ListT5: Listwise Reranking with Fusion-in-Decoder Improves Zero-shot Retrieval](https://mlsanity.com/paper/2402.15838/) (2024)
- [RAGAR, Your Falsehood Radar: RAG-Augmented Reasoning for Political Fact-Checking using Multimodal Large Language Models](https://mlsanity.com/paper/2404.12065/) (2024)
- [To Trust or Not to Trust? Enhancing Large Language Models' Situated Faithfulness to External Contexts](https://mlsanity.com/paper/2410.14675/) (2024)
- [RAG Does Not Work for Enterprises](https://mlsanity.com/paper/2406.04369/) (2024)
- [Semantically Constrained Memory Allocation (SCMA) for Embedding in Efficient Recommendation Systems](https://mlsanity.com/paper/2103.06124/) (2021)
- [A novel multi-threaded web crawling model](https://mlsanity.com/paper/2407.10440/) (2024)

## Built on by (most cited first, 14 of 14)

- [Benchmarking Poisoning Attacks against Retrieval-Augmented Generation](https://mlsanity.com/paper/2505.18543/) (2025)
- [CPA-RAG:Covert Poisoning Attacks on Retrieval-Augmented Generation in Large Language Models](https://mlsanity.com/paper/2505.19864/) (2025)
- [MIRAGE: Misleading Retrieval-Augmented Generation via Black-box and Query-agnostic Poisoning Attacks](https://mlsanity.com/paper/2512.08289/) (2025)
- [Through the Stealth Lens: Attention-Aware Defenses Against Poisoning in RAG](https://mlsanity.com/paper/2506.04390/) (2025)
- [Securing Retrieval-Augmented Generation: A Taxonomy of Attacks, Defenses, and Future Directions](https://mlsanity.com/paper/2604.08304/) (2026)
- [Inference Cost Attacks for Retrieval-Augmented Large Language Models](https://mlsanity.com/paper/2606.02643/) (2026)
- [Conflict-Aware Retriever Editing for Knowledge Injection Attacks on LLM-Based RAG Systems](https://mlsanity.com/paper/2606.18310/) (2026)
- [Retrieved But Not Reliable: A Survey on Attacks, and Defenses in Retrieval-Augmented Generation](https://mlsanity.com/paper/2608.24977/) (2026)
- [Quantifying Document Impact in RAG-LLMs](https://mlsanity.com/paper/2601.05260/) (2025)
- [When Poison Fails After Retrieval: Revisiting Corpus Poisoning under Chunking and Reranking Pipelines](https://mlsanity.com/paper/2606.11265/) (2026)
- [KidnapRAG: A Black-Box Attack for Hijacking Reasoning in Agentic Retrieval-Augmented Generation Systems](https://mlsanity.com/paper/2607.00422/) (2026)
- [Improving Rare Medication Recommendation with Counterfactual Data Augmentation and Large Language Models](https://mlsanity.com/paper/2607.24829/) (2026)
- [RAGSieve: Self-Referenced Local Contrast for Knowledge-Poison Detection in Retrieval-Augmented Generation](https://mlsanity.com/paper/2608.13010/) (2026)
- [Divide and Doubt: Diverse Distributed Poisoning for Retrieval-Augmented Generation](https://mlsanity.com/paper/2609.27090/) (2026)

## Abstract (arXiv)

With the growing adoption of retrieval-augmented generation (RAG) systems, various attack methods have been proposed to degrade their performance. However, most existing approaches rely on unrealistic assumptions in which external attackers have access to internal components such as the retriever. To address this issue, we introduce a realistic black-box attack based on the RAG paradox, a structural vulnerability arising from the system's effort to enhance trust by revealing both the retrieved documents and their sources to users. This transparency enables attackers to observe which sources are used and how information is phrased, allowing them to craft poisoned documents that are more likely to be retrieved and upload them to the identified sources. Moreover, as RAG systems directly provide retrieved content to users, these documents must not only be retrievable but also appear natural and credible to maintain user confidence in the search results. Unlike prior work that focuses solely on improving document retrievability, our attack method explicitly considers both retrievability and user trust in the retrieved content. Both offline and online experiments demonstrate that our method significantly degrades system performance without internal access, while generating natural-looking poisoned documents.