# Prompt injection and agent security Area: [LLM safety and security](https://mlsanity.com/safety/index.md). HTML: https://mlsanity.com/safety/prompt-injection-and-agent-security/ ## Key facts - Prompt injection and agent security: 959 papers on arXiv since 2021 (data through 14 Oct 2026). - 347 of its 959 mapped papers (36%) cite “Not what you've signed up for: Compromising Real-World LLM-Integrated Applications with Indirect Prompt Injection” (2023, arXiv 2302.12173). - 172 of its 959 mapped papers (18%) cite “AgentDojo: A Dynamic Environment to Evaluate Prompt Injection Attacks and Defenses for LLM Agents” (2024, arXiv 2406.13352). - 60 new papers 24 Sep–14 Oct. ## Most cited here - [Not what you've signed up for: Compromising Real-World LLM-Integrated Applications with Indirect Prompt Injection](https://mlsanity.com/paper/2302.12173/) (2023), cited by 347 papers here - [AgentDojo: A Dynamic Environment to Evaluate Prompt Injection Attacks and Defenses for LLM Agents](https://mlsanity.com/paper/2406.13352/) (2024), cited by 172 papers here - [StruQ: Defending Against Prompt Injection with Structured Queries](https://mlsanity.com/paper/2402.06363/) (2024), cited by 170 papers here - [InjecAgent: Benchmarking Indirect Prompt Injections in Tool-Integrated Large Language Model Agents](https://mlsanity.com/paper/2403.02691/) (2024), cited by 168 papers here - [Prompt Injection attack against LLM-integrated Applications](https://mlsanity.com/paper/2306.05499/) (2023), cited by 148 papers here - [Formalizing and Benchmarking Prompt Injection Attacks and Defenses](https://mlsanity.com/paper/2310.12815/) (2023), cited by 143 papers here - [HarmBench: A Standardized Evaluation Framework for Automated Red Teaming and Robust Refusal](https://mlsanity.com/paper/2402.04249/) (2024), cited by 133 papers here - [The Instruction Hierarchy: Training LLMs to Prioritize Privileged Instructions](https://mlsanity.com/paper/2404.13208/) (2024), cited by 131 papers here - [Red Teaming Language Models with Language Models](https://mlsanity.com/paper/2202.03286/) (2022), cited by 129 papers here - [Defending Against Indirect Prompt Injection Attacks With Spotlighting](https://mlsanity.com/paper/2403.14720/) (2024), cited by 118 papers here ## New (24 Sep–14 Oct) - [StepSafe: Learning Timely Intervention with Context-Aware Safety Reasoning](https://mlsanity.com/paper/2610.20960/) (2026-10-14) - [Screwtape: Adversarial Transcript Editing for Monitor Red-Teaming](https://mlsanity.com/paper/2610.20941/) (2026-10-14) - [INFUSE: A Unified Surface Benchmark for Indirect Prompt Injection in Agentic Systems](https://mlsanity.com/paper/2610.20776/) (2026-10-14) - [Are You Reading the Right Thing? Rethinking Internal Signals for Prompt-Injection Detection](https://mlsanity.com/paper/2610.19452/) (2026-10-14) - [AgentSafeEvo: Adversarial Co-Evolution for LLM Agent Safety](https://mlsanity.com/paper/2610.19422/) (2026-10-14) - [TRACER: Trace-Regularized Training Against Indirect Prompt Injection](https://mlsanity.com/paper/2610.19133/) (2026-10-13) - [Towards Safe Autonomy for Computer Use](https://mlsanity.com/paper/2610.18508/) (2026-10-13) - [Jev-Guard: Fast Routing and Auditing Agent Actions over Editable Policy Trees](https://mlsanity.com/paper/2610.18326/) (2026-10-13) - [BrowseSafe++: Benchmarking Prompt Injection Detection Across Web Contexts, Languages, and Browser Views](https://mlsanity.com/paper/2610.17457/) (2026-10-12) - [Same Operation, Different Permission: Evaluating Whether Guardrails Honor Authorization](https://mlsanity.com/paper/2610.16838/) (2026-10-12) - [REPRO: Reflective Evolutionary Prompt Optimizer for Automated LLM Red-Teaming](https://mlsanity.com/paper/2610.15513/) (2026-10-11) - [EvolvAttack: Backdooring self Evolving VLM Agents](https://mlsanity.com/paper/2610.15244/) (2026-10-11) - [Only What You've Signed up For: Secure Workflows with Constrained Diffusion Agents](https://mlsanity.com/paper/2610.15177/) (2026-10-10) - [X-AgentRed: Red-Teaming Personal Agents in Evolving Multi-Service Ecosystems](https://mlsanity.com/paper/2610.13886/) (2026-10-09) - [VEPA: Verified Evolving Formal Policy for Lifelong LLM Agent Guardrails](https://mlsanity.com/paper/2610.12813/) (2026-10-08) - [One Word Opens the Gate: The Option-Channel Attack on Typed Decision Models as Agent Guardrails](https://mlsanity.com/paper/2610.12292/) (2026-10-08) - [ReSI: Recursive Safety Improvement toward Resistant and Resilient AI](https://mlsanity.com/paper/2610.12233/) (2026-10-08) - [LTBD: Learnable Trust-Boundary Delimiters for Prompt Injection Defense](https://mlsanity.com/paper/2610.11634/) (2026-10-08) - [BRANCH: Bypassing Multi-Scanner AI Guardrails](https://mlsanity.com/paper/2610.10742/) (2026-10-07) - [Constrained-Action AI Remediation for SIEM/XDR via a NeMo-Guardrails Proxy](https://mlsanity.com/paper/2610.09906/) (2026-10-07) - [Defensive Sufficiency in a Stackelberg Model of AI Security](https://mlsanity.com/paper/2610.09892/) (2026-10-07) - [Package Hallucination Attacks on Coding Agents through Prompt Injection in Rule Files](https://mlsanity.com/paper/2610.09264/) (2026-10-07) - [Adversarial Images Hijack Web Agents from Visual Grounding to Browser Execution](https://mlsanity.com/paper/2610.09240/) (2026-10-07) - [AdvSim2Real : Training Web Agents Against Adaptive Prompt Injection in a Web World Model](https://mlsanity.com/paper/2610.08773/) (2026-10-06) - [POLAR: Ontology-Guided Risk Prevention for Tool-Calling LLM Agents](https://mlsanity.com/paper/2610.08082/) (2026-10-06) - [Towards a Unified Misuse Monitoring Benchmark](https://mlsanity.com/paper/2610.07089/) (2026-10-05) - [RAISED: Self-Distillation for Robustness to Prompt Injection in LLM Agents](https://mlsanity.com/paper/2610.06401/) (2026-10-05) - [Benchmarking Jailbreak Guardrails for Embodied Agents](https://mlsanity.com/paper/2610.06122/) (2026-10-05) - [Topology-Conditioned Backdoors: Language Models That Insert Vulnerabilities When They Infer They Are in a Multi-Agent System](https://mlsanity.com/paper/2610.05793/) (2026-10-05) - [Reflections and Fragments: Securing LLMs Against Sequential Mosaic Attacks](https://mlsanity.com/paper/2610.05346/) (2026-10-04)