mlsanityData to 7 Oct 2026

Overcoming the Retrieval Barrier: Indirect Prompt Injection in the Wild for LLM Systems

arXiv 2601.07072 ↗First posted 11 Jan 2026cs.CR cs.AIcited by 16

On the map: LLM safety and security → Prompt injection and agent security

Abstract (arXiv)

Large language models (LLMs) increasingly rely on retrieving information from external corpora. This creates a new attack surface: indirect prompt injection (IPI), where hidden instructions are planted in the corpora and hijack model behavior once retrieved. Previous studies have highlighted this risk but often avoid the hardest step: ensuring that malicious content is actually retrieved. In practice, unoptimized IPI is rarely retrieved under natural queries, which leaves its real-world impact unclear. We address this challenge by decomposing the malicious content into a trigger fragment that guarantees retrieval and an attack fragment that encodes arbitrary attack objectives. Based on this idea, we design an efficient and effective black-box attack algorithm that constructs a compact trigger fragment to guarantee retrieval for any attack fragment. Our attack requires only API access to embedding models, is cost-efficient (as little as 0.21 per target user query on OpenAI's embedding models), and achieves near-100% retrieval across 11 benchmarks and 8 embedding models (including both open-source models and proprietary services). Based on this attack, we present the first end-to-end IPI exploits under natural queries and realistic external corpora, spanning both RAG and agentic systems with diverse attack objectives. These results establish IPI as a practical and severe threat: when a user issued a natural query to summarize emails on frequently asked topics, a single poisoned email was sufficient to coerce GPT-4o into exfiltrating SSH keys with over 80% success in a multi-agent workflow. We further evaluate several defenses and find that they are insufficient to prevent the retrieval of malicious text, highlighting retrieval as a critical open vulnerability.

Key facts. Cited by 16 papers in the map (data through 7 Oct 2026). Placed in Prompt injection and agent security, one of 12 directions in LLM safety and security. Builds on 80 of its 126 references that are in the map. First posted on arXiv 11 Jan 2026.

Builds on

80 of 126 references.

Closest ancestors · 18

Same direction, newest first

  1. 2025PromptArmor: Simple yet Effective Prompt Injection Defenses21 Jul
  2. 2025LLMail-Inject: A Dataset from a Realistic Adaptive Prompt Injection Challenge11 Jun
  3. 2025Lessons from Defending Gemini Against Indirect Prompt Injections20 May
  4. 2025AgentVigil: Generic Black-Box Red-teaming for Indirect Prompt Injection against LLM Agents9 May
  5. 2025DataSentinel: A Game-Theoretic Detection of Prompt Injection Attacks15 Apr
  6. 2024FATH: Authentication-based Test-time Defense against Indirect Prompt Injection Attacks28 Oct
  7. 2024SecAlign: Defending Against Prompt Injection with Preference Optimization7 Oct
  8. 2024Agent Security Bench (ASB): Formalizing and Benchmarking Attacks and Defenses in LLM-based Agents3 Oct
  9. 2024AgentPoison: Red-teaming LLM Agents via Poisoning Memory or Knowledge Bases17 Jul
  10. 2024Get my drift? Catching LLM Task Drift with Activation Deltas2 Jun
  11. 2024Optimization-based Prompt Injection Attack to LLM-as-a-Judge26 Mar
  12. 2024Neural Exec: Learning (and Learning from) Execution Triggers for Prompt Injection Attacks6 Mar
6 more
  1. 2024Here Comes The AI Worm: Unleashing Zero-click Worms that Target GenAI-Powered Applications5 Mar
  2. 2024InjecAgent: Benchmarking Indirect Prompt Injections in Tool-Integrated Large Language Model Agents5 Mar
  3. 2024StruQ: Defending Against Prompt Injection with Structured Queries9 Feb
  4. 2023Formalizing and Benchmarking Prompt Injection Attacks and Defenses19 Oct
  5. 2023Prompt Injection attack against LLM-integrated Applications8 Jun
  6. 2023Not what you've signed up for: Compromising Real-World LLM-Integrated Applications with Indirect Prompt Injection23 Feb

Other references · 62

  1. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,160
  2. 2020Dense Passage Retrieval for Open-Domain Question Answeringcited by 3,779
  3. 2018HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answeringcited by 3,686
  4. 2021Convex optimizationcited by 2,479
  5. 2020REALM: Retrieval-Augmented Language Model Pre-Trainingcited by 2,126
  6. 2023AutoGen: Enabling Next-Gen LLM Applications via Multi-Agent Conversationcited by 1,906
  7. 2015Microsoft COCO Captions: Data Collection and Evaluation Servercited by 1,447
  8. 2018FEVER: a large-scale dataset for Fact Extraction and VERificationcited by 1,362
  9. 2021BEIR: A Heterogenous Benchmark for Zero-shot Evaluation of Information Retrieval Modelscited by 1,225
  10. 2021Unsupervised Dense Information Retrieval with Contrastive Learningcited by 1,203
  11. 2025Qwen3 Embedding: Advancing Text Embedding and Reranking Through Foundation Modelscited by 1,175
  12. 2022Reproducible scaling laws for contrastive language-image learningcited by 1,163
38 more
  1. 2022Ignore Previous Prompt: Attack Techniques For Language Modelscited by 760
  2. 2023AutoDAN: Generating Stealthy Jailbreak Prompts on Aligned Large Language Modelscited by 738
  3. 2023Towards General Text Embeddings with Multi-stage Contrastive Learningcited by 705
  4. 2019Passage Re-ranking with BERTcited by 629
  5. 2023Baseline Defenses for Adversarial Attacks Against Aligned Language Modelscited by 608
  6. 2020Fact or Fiction: Verifying Scientific Claimscited by 546
  7. 2023SmoothLLM: Defending Large Language Models Against Jailbreaking Attackscited by 434
  8. 2020SPECTER: Document-level Representation Learning using Citation-informed Transformerscited by 426
  9. 2023Detecting Language Model Attacks with Perplexitycited by 411
  10. 2023Poisoning Web-Scale Training Datasets is Practicalcited by 276
  11. 2024mGTE: Generalized Long-Context Text Representation and Reranking Models for Multilingual Text Retrievalcited by 233
  12. 2024Magentic-One: A Generalist Multi-Agent System for Solving Complex Taskscited by 221
  13. 2022Demystifying Prompts in Language Models via Perplexity Estimationcited by 215
  14. 2023Poisoning Retrieval Corpora by Injecting Adversarial Passagescited by 153
  15. 2023On the Risk of Misinformation Pollution with Large Language Modelscited by 124
  16. 2024Machine Against the RAG: Jamming Retrieval-Augmented Generation with Blocker Documentscited by 93
  17. 2024BadRAG: Identifying Vulnerabilities in Retrieval Augmented Generation of Large Language Modelscited by 84
  18. 2024Follow My Instruction and Spill the Beans: Scalable Data Extraction from Retrieval-Augmented Generation Systemscited by 76
  19. 2025Multi-Agent Systems Execute Arbitrary Malicious Codecited by 52
  20. 2022Synthetic Disinformation Attacks on Automated Fact Verification Systemscited by 44
  21. 2022PRADA: Practical Black-Box Adversarial Attacks against Neural Ranking Modelscited by 44
  22. 2020Adversarial Semantic Collisionscited by 42
  23. 2022Order-Disorder: Imitation Adversarial Attacks for Black-box Neural Ranking Modelscited by 37
  24. 2023Black-box Adversarial Attacks against Dense Retrieval Models: A Multi-view Contrastive Learning Methodcited by 29
  25. 2024HijackRAG: Hijacking Attacks against Retrieval-Augmented Large Language Modelscited by 27
  26. 2024To Trust or Not to Trust? Enhancing Large Language Models' Situated Faithfulness to External Contextscited by 25
  27. 2020One word at a time: adversarial attacks on retrieval modelscited by 23
  28. 2024SciQAG: A Framework for Auto-Generated Science Question Answering Dataset with Fine-grained Evaluationcited by 23
  29. 2024Backdoored Retrievers for Prompt Injection Attacks on Retrieval Augmented Generation of Large Language Modelscited by 21
  30. 2023Topic-oriented Adversarial Attacks against Black-box Neural Ranking Modelscited by 18
  31. 2023Towards Imperceptible Document Manipulations against Neural Ranking Modelscited by 18
  32. 2022BERT Rankers are Brittle: a Study using Adversarial Document Perturbationscited by 16
  33. 2024Advancing the Robustness of Large Language Models through Self-Denoised Smoothingcited by 15
  34. 2024Rag and Roll: An End-to-End Evaluation of Indirect Prompt Manipulations in LLM-based Application Frameworkscited by 14
  35. 2024Black-Box Opinion Manipulation Attacks to Retrieval-Augmented Generation of Large Language Modelscited by 11
  36. 2025Invitation Is All You Need! Promptware Attacks Against LLM-Powered Assistants in Production Are Practical and Dangerouscited by 11
  37. 2025The Silent Saboteur: Imperceptible Adversarial Attacks against Black-Box Retrieval-Augmented Generation Systemscited by 10
  38. 2024Beyond-RAG: Question Identification and Answer Generation in Real-Time Conversationscited by 9

50 most cited of 62 All 62 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 16.

16 papers.

Sort:

By area:

  1. 2026ER-MIA: Black-Box Adversarial Memory Injection Attacks on Long-Term Memory-Augmented Large Language Modelscited by 4
  2. 2026DiscourseFlip: An Oblique Discourse-Level Opinion Manipulation Attack against Black-box Retrieval-Augmented Generationcited by 1
  3. 2026PI-Hunter: Automated Red-Teaming for Exposing and Localizing Prompt Injectionscited by 1
  4. 2026TrustedARI: Towards Trust-Native Agentic Routing Infrastructure for Agentic AIcited by 1
  5. 2026Isolation as a First-Class Principle for LLM-Agent System Safety: Concepts, Taxonomy, Challenges and Future Directionscited by 1
  6. 2026Blockchain Empowered Trustworthy Agent Networks: Foundations, Taxonomy, and Future Directionscited by 1
  7. 2026Misleading the Planner through Deceptive Resumes: Registration-Time Injection in Centralized Multi-Agent Systemscited by 1
  8. 2026DualView: Preventing Indirect Prompt Injection in Personal AI Agentscited by 0
  9. 2026RAGSieve: Self-Referenced Local Contrast for Knowledge-Poison Detection in Retrieval-Augmented Generationcited by 0
  10. 2026WebMCP-Phalanx: Enforcing and Characterizing Trust Boundaries for Browser-Integrated LLM Agentscited by 0
  11. 2026Influence Is Not Authority: When Causal Guardrail Signals Make Legitimate Tool Use Look Like an Attack in Tool-Using LLM Agentscited by 0
  12. 2026ActGuard: Pre-execution Action Auditing against Indirect Prompt Injection in LLM Agentscited by 0
4 more
  1. 2026CiteShade: Citation Laundering in Multi-Source Retrieval-Augmented Generation and Its Counterfactual Defensecited by 0
  2. 2026Connecting the Dots in Agentic AI Security: A Cross-Dimensional Threat Taxonomy, Evaluation Maturity, and Open Challengescited by 0
  3. 2026CoDeL: Co-Evolutionary Defense against Indirect Prompt Injection in LLM-based Agentscited by 0
  4. 2026Chaining Skills to Hijack LLM Agentscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Exploiting Web Search Tools of AI Agents for Data Exfiltration
  2. 2026"Someone Hid It": Query-Agnostic Black-Box Attacks on LLM-Based Retrieval
  3. 2024Data Extraction Attacks in Retrieval-Augmented Generation via Backdoors
  4. 2026A Layered Security Framework Against Prompt Injection in RAG-Based Chatbots
  5. 2025Attention is All You Need to Defend Against Indirect Prompt Injection Attacks in LLMs
  6. 2026PIDP-Attack: Combining Prompt Injection with Database Poisoning Attacks on Retrieval-Augmented Generation Systems
  7. 2025External Data Extraction Attacks against Retrieval-Augmented Large Language Models
  8. 2026Inference Cost Attacks for Retrieval-Augmented Large Language Models