mlsanityData to 7 Oct 2026

Human-Imperceptible Retrieval Poisoning Attacks in LLM-Powered Applications

arXiv 2404.17196 ↗First posted 26 Apr 2024cs.CR cs.AIcited by 32

On the map: Retrieval-augmented generation → Privacy and security of RAG

Abstract (arXiv)

Presently, with the assistance of advanced LLM application development frameworks, more and more LLM-powered applications can effortlessly augment the LLMs' knowledge with external content using the retrieval augmented generation (RAG) technique. However, these frameworks' designs do not have sufficient consideration of the risk of external content, thereby allowing attackers to undermine the applications developed with these frameworks. In this paper, we reveal a new threat to LLM-powered applications, termed retrieval poisoning, where attackers can guide the application to yield malicious responses during the RAG process. Specifically, through the analysis of LLM application frameworks, attackers can craft documents visually indistinguishable from benign ones. Despite the documents providing correct information, once they are used as reference sources for RAG, the application is misled into generating incorrect responses. Our preliminary experiments indicate that attackers can mislead LLMs with an 88.33% success rate, and achieve a 66.67% success rate in the real-world application, demonstrating the potential impact of retrieval poisoning.

Key facts. Cited by 32 papers in the map (data through 7 Oct 2026). Placed in Privacy and security of RAG, one of 15 directions in Retrieval-augmented generation. Builds on 13 of its 24 references that are in the map. First posted on arXiv 26 Apr 2024.

Builds on

13 of 24 references.

References · 13

  1. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,865
  2. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,160
  3. 2023Universal and Transferable Adversarial Attacks on Aligned Language Modelscited by 2,914
  4. 2023Not what you've signed up for: Compromising Real-World LLM-Integrated Applications with Indirect Prompt Injectioncited by 1,339
  5. 2023Jailbreaking Black Box Large Language Models in Twenty Queriescited by 1,326
  6. 2020Leveraging Passage Retrieval with Generative Models for Open Domain Question Answeringcited by 1,203
  7. 2020MPNet: Masked and Permuted Pre-training for Language Understandingcited by 973
  8. 2023Prompt Injection attack against LLM-integrated Applicationscited by 568
  9. 2023Catastrophic Jailbreak of Open-source LLMs via Exploiting Generationcited by 439
  10. 2019Neural Machine Translation: A Review and Surveycited by 99
  11. 2021Colossal-AI: A Unified Deep Learning System For Large-Scale Parallel Trainingcited by 96
  12. 2023Prompting Frameworks for Large Language Models: A Surveycited by 29
1 more
  1. 2020On the Transferability of Adversarial Attacksagainst Neural Text Classifiercited by 16

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 6; 2025: 17; 2026: 9.

32 papers.

Sort:

By area:

  1. 2025Safety at Scale: A Comprehensive Survey of Large Model and Agent Safetycited by 63
  2. 2025From Prompt Injections to Protocol Exploits: Threats in LLM-Powered AI Agents Workflowscited by 57
  3. 2025PR-Attack: Coordinated Prompt-RAG Attacks on Retrieval-Augmented Generation in Large Language Models via Bilevel Optimizationcited by 29
  4. 2025Agentic AI Security: Threats, Defenses, Evaluation, and Open Challengescited by 26
  5. 2025FlippedRAG: Black-Box Opinion Manipulation Adversarial Attacks to Retrieval-Augmented Generation Modelscited by 24
  6. 2024Adversarial Decoding: Generating Readable Documents for Adversarial Objectivescited by 16
  7. 2024SafeEmbodAI: a Safety Framework for Mobile Robots in Embodied AI Systemscited by 12
  8. 2025Poisoned-MRAG: Knowledge Poisoning Attacks to Multimodal Retrieval Augmented Generationcited by 12
  9. 2025Prompt Injection 2.0: Hybrid AI Threatscited by 12
  10. 2024Black-Box Opinion Manipulation Attacks to Retrieval-Augmented Generation of Large Language Modelscited by 11
  11. 2024The Dark Side of Trust: Authority Citation-Driven Jailbreak Attacks on Large Language Modelscited by 9
  12. 2026SoK: Privacy Risks and Mitigations in Retrieval-Augmented Generation Systemscited by 7
20 more
  1. 2025POISONCRAFT: Practical Poisoning of Retrieval-Augmented Generation for Large Language Modelscited by 6
  2. 2025Understanding the Anchoring Effect of LLM with Synthetic Data: Existence, Mechanism, and Potential Mitigationscited by 6
  3. 2025Towards Advancing Code Generation with Large Language Models: A Research Roadmapcited by 5
  4. 2025Uncovering Competing Poisoning Attacks in Retrieval-Augmented Generationcited by 5
  5. 2025Exploring Backdoor Attack and Defense for LLM-empowered Recommendationscited by 4
  6. 2026Evaluating Privilege Usage of Agents with Real-World Toolscited by 4
  7. 2025OMNISEC: LLM-Driven Provenance-based Intrusion Detection via Retrieval-Augmented Behavior Promptingcited by 3
  8. 2025A First Look at the Security Issues in the Model Context Protocol Ecosystemcited by 3
  9. 2026Architecting Trust in Artificial Epistemic Agentscited by 3
  10. 2024The Best Defense is a Good Offense: Countering LLM-Powered Cyberattackscited by 2
  11. 2024On the Validity of Traditional Vulnerability Scoring Systems for Adversarial Attacks against LLMscited by 2
  12. 2025Medusa: Cross-Modal Transferable Adversarial Attacks on Multimodal Medical Retrieval-Augmented Generationcited by 2
  13. 2026Securing Retrieval-Augmented Generation: A Taxonomy of Attacks, Defenses, and Future Directionscited by 2
  14. 2025Layer-Wise Perturbations via Sparse Autoencoders for Adversarial Text Generationcited by 1
  15. 2025A Survey: Towards Privacy and Security in Mobile Large Language Modelscited by 1
  16. 2026LLM-Oriented Information Retrieval: A Denoising-First Perspectivecited by 1
  17. 2026Needle-in-RAG: Prompt-Conditioned Character-Level Traceback of Poisoned Spans in Retrieved Evidencecited by 1
  18. 2026Isolation as a First-Class Principle for LLM-Agent System Safety: Concepts, Taxonomy, Challenges and Future Directionscited by 1
  19. 2026"Are You Sure?": An Empirical Study of Human Perception Vulnerability in LLM-Driven Agentic Systemscited by 0
  20. 2026Attacking and Defending Multi-Agent Collaborative Filtering Systems Through Connectivitycited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024Rag and Roll: An End-to-End Evaluation of Indirect Prompt Manipulations in LLM-based Application Frameworks
  2. 2025Practical Poisoning Attacks against Retrieval-Augmented Generation
  3. 2024RevPRAG: Revealing Poisoning Attacks in Retrieval-Augmented Generation through LLM Activation Analysis
  4. 2026Influence Factors on RAG Poisoning
  5. 2025The RAG Paradox: A Black-Box Attack Exploiting Unintentional Vulnerabilities in Retrieval-Augmented Generation Systems
  6. 2025Benchmarking Poisoning Attacks against Retrieval-Augmented Generation
  7. 2026Confundo: Learning to Generate Robust Poison for Practical RAG Systems
  8. 2025Secure Retrieval-Augmented Generation against Poisoning Attacks