# AgentDyn: Are Your Agent Security Defenses Deployable in Real-World Dynamic Environments?

arXiv: https://arxiv.org/abs/2602.03117 · HTML: https://mlsanity.com/paper/2602.03117/

## Key facts

- Cited by 59 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 3 Feb 2026.

## Built on by (most cited first, 30 of 59)

- [ClawSafety: "Safe" LLMs, Unsafe Agents](https://mlsanity.com/paper/2604.01438/) (2026)
- [PIArena: A Platform for Prompt Injection Evaluation](https://mlsanity.com/paper/2604.08499/) (2026)
- [Architecting Secure AI Agents: Perspectives on System-Level Defenses Against Indirect Prompt Injection Attacks](https://mlsanity.com/paper/2603.30016/) (2026)
- [HINTBench: Horizon-agent Intrinsic Non-attack Trajectory Benchmark](https://mlsanity.com/paper/2604.13954/) (2026)
- [AgentDoG 1.5: A Lightweight and Scalable Alignment Framework for AI Agent Safety and Security](https://mlsanity.com/paper/2605.29801/) (2026)
- [Assessing Automated Prompt Injection Attacks in Agentic Environments](https://mlsanity.com/paper/2606.10525/) (2026)
- [Toward Secure LLM Agents: Threat Surfaces, Attacks, Defenses, and Evaluation](https://mlsanity.com/paper/2606.10749/) (2026)
- [Your Agentic LLMs Secretly Encode Indirect Prompt-Injection Exposure in Hidden States](https://mlsanity.com/paper/2608.02657/) (2026)
- [AgentWatcher: A Rule-based Prompt Injection Monitor](https://mlsanity.com/paper/2604.01194/) (2026)
- [No Attacker Needed: Unintentional Cross-User Contamination in Shared-State LLM Agents](https://mlsanity.com/paper/2604.01350/) (2026)
- [SABER: Benchmarking Operational Safety of LLM Coding Agents in Stateful Project Workspaces](https://mlsanity.com/paper/2606.01317/) (2026)
- [StepGuard: Learning Step-Level Guardrails with Scalable Supervision and Safety-Utility Balancing](https://mlsanity.com/paper/2608.24777/) (2026)
- [SafeEvolve: Harness-Policy Co-Evolution from Agent Experience for Safety Alignment](https://mlsanity.com/paper/2609.02786/) (2026)
- [AutoDojo: Adaptive Black-Box Attacks Reveal the Limits of IPI Defenses and Task-Specification Effects in LLM Agents](https://mlsanity.com/paper/2606.15057/) (2026)
- [Intent-Governed Tool Authorization for AI Agents](https://mlsanity.com/paper/2606.22916/) (2026)
- [Your Agent is More Brittle Than You Think: Uncovering Indirect Injection Vulnerabilities in Agentic LLMs](https://mlsanity.com/paper/2604.03870/) (2026)
- [An AI Agent Execution Environment to Safeguard User Data](https://mlsanity.com/paper/2604.19657/) (2026)
- [NetInjectBench: Benchmarking Indirect Prompt Injection in Tool-Using Large Language Model Agents for Network Operations](https://mlsanity.com/paper/2607.10490/) (2026)
- [Adaptive Evaluation of Out-of-Band Defenses Against Prompt Injection in LLM Agents](https://mlsanity.com/paper/2606.26479/) (2026)
- [Permission Denied: Policy-Graded Evaluation of Coding Agents in Hardened Environments](https://mlsanity.com/paper/2608.02670/) (2026)
- [Hallucination as Exploit: Evidence-Carrying Multimodal Agents](https://mlsanity.com/paper/2605.19192/) (2026)
- [Who Pays the Price? Stakeholder-Centric Prompt Injection Benchmarking for Real-world Web Agents](https://mlsanity.com/paper/2606.13385/) (2026)
- [Agent Hacks Agent: Autoresearch for Production-Agent Red-Teaming](https://mlsanity.com/paper/2607.11698/) (2026)
- [SingGuard-NSFA: Extensible Guardrails for Agentic AI via Generative Reasoning and Real-Time Classification](https://mlsanity.com/paper/2607.13081/) (2026)
- [ROPE: Routed Origin Policy Enforcement against Indirect Prompt Injection](https://mlsanity.com/paper/2608.27496/) (2026)
- [EvoSafeHarness: Evolving Model- and Domain-Specific Harnesses for Securing Agents](https://mlsanity.com/paper/2609.05903/) (2026)
- [AgentDrift: A Step-Labeled Benchmark of Injection-Hijacked LLM Agent Trajectories](https://mlsanity.com/paper/2609.06972/) (2026)
- [CoER: Defending against Adaptive Indirect Prompt Injection via Adversarial Co-Evolution and Refinement](https://mlsanity.com/paper/2609.07529/) (2026)
- [Security of OpenClaw Agents: Fundamentals, Attacks, and Countermeasures](https://mlsanity.com/paper/2605.25435/) (2026)
- [ContainmentBench: Trace-Based Evaluation of Post-Exposure Containment in Tool-Using LLM Agents](https://mlsanity.com/paper/2607.23999/) (2026)

## Abstract (arXiv)

AI agents that autonomously interact with external tools and environments have shown great promise across real-world applications. However, their reliance on external data exposes them to serious indirect prompt injection attacks, where malicious instructions embedded in third-party content hijack agent behaviors. To mitigate this threat, a growing number of defenses have been proposed and evaluated under existing agent security benchmarks. These benchmarks provide structured environments for comparing attacks and defenses, and have become a key driver for defense design and optimization. However, as agents move toward more complex and open-ended real-world deployments, there is a pressing need for benchmarks to become more adaptive and better reflect the dynamic environments faced by real-world agentic systems. In this work, we reveal three fundamental flaws in the current benchmarks and push the frontier along these dimensions: (i) lack of dynamic open-ended tasks, (ii) lack of helpful instructions, and (iii) simplistic user tasks. To bridge this gap, we introduce AgentDyn, a manually designed benchmark featuring 60 challenging open-ended tasks and 560 injection test cases across Shopping, GitHub, and Daily Life. Unlike prior static benchmarks, AgentDyn requires dynamic planning and incorporates helpful third-party instructions. Our evaluation of ten state-of-the-art defenses suggests that almost all existing defenses are either not secure enough or suffer from significant over-defense, revealing that existing defenses are still far from real-world deployment. Our benchmark is available at https://github.com/leolee99/AgentDyn.