mlsanityData to 7 Oct 2026

MUZZLE: Adaptive Agentic Red-Teaming of Web Agents Against Indirect Prompt Injection Attacks

arXiv 2602.09222 ↗First posted 9 Feb 2026cs.CR cs.AIcited by 18

On the map: LLM agents and tool use → Agent threats, guardrails and governance

also in 1 more
Abstract (arXiv)

Large language model (LLM) based web agents are increasingly deployed to automate complex online tasks by directly interacting with web sites and performing actions on users' behalf. While these agents offer powerful capabilities, their design exposes them to indirect prompt injection attacks embedded in untrusted web content, enabling adversaries to hijack agent behavior and violate user intent. Despite growing awareness of this threat, existing evaluations rely on fixed attack templates, manually selected injection surfaces, or narrowly scoped scenarios, limiting their ability to capture realistic, adaptive attacks encountered in practice. We present MUZZLE, an automated agentic framework for evaluating the security of web agents against indirect prompt injection attacks. MUZZLE utilizes the agent's trajectories to automatically identify high-salience injection surfaces, and adaptively generate context-aware malicious instructions that target violations of confidentiality, integrity, and availability. Unlike prior approaches, MUZZLE adapts its attack strategy based on the agent's observed execution trajectory and iteratively refines attacks using feedback from failed executions. We evaluate MUZZLE across diverse web applications, user tasks, and agent configurations, demonstrating its ability to automatically and adaptively assess the security of web agents with minimal human intervention. Our results show that MUZZLE effectively discovers 44 new attacks on 4 web applications with 10 adversarial objectives that violate confidentiality, availability, or privacy properties across different LLMs and agent scaffolds. MUZZLE also identifies novel attack strategies, including 3 cross-application prompt injection attacks and an agent-tailored phishing scenario.

Key facts. Cited by 18 papers in the map (data through 7 Oct 2026). Placed in Agent threats, guardrails and governance, one of 16 directions in LLM agents and tool use. Builds on 50 of its 89 references that are in the map. First posted on arXiv 9 Feb 2026.

Builds on

50 of 89 references.

Closest ancestors · 11

Same direction, newest first

  1. 2025RedCodeAgent: Automatic Red-teaming Agent against Diverse Code Agents2 Oct
  2. 2025OS-Harm: A Benchmark for Measuring Safety of Computer Use Agents17 Jun
  3. 2025AgentVigil: Generic Black-Box Red-teaming for Indirect Prompt Injection against LLM Agents9 May
  4. 2025WASP: Benchmarking Web Agent Security Against Prompt Injection Attacks22 Apr
  5. 2025MELON: Provable Defense Against Indirect Prompt Injection Attacks in AI Agents7 Feb
  6. 2024AdvAgent: Controllable Blackbox Red-teaming on Web Agents22 Oct
  7. 2024AgentHarm: A Benchmark for Measuring Harmfulness of LLM Agents11 Oct
  8. 2024Agent Security Bench (ASB): Formalizing and Benchmarking Attacks and Defenses in LLM-based Agents3 Oct
  9. 2024AgentPoison: Red-teaming LLM Agents via Poisoning Memory or Knowledge Bases17 Jul
  10. 2024Dissecting Adversarial Robustness of Multimodal LM Agents18 Jun
  11. 2024InjecAgent: Benchmarking Indirect Prompt Injections in Tool-Integrated Large Language Model Agents5 Mar

Other references · 39

  1. 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,355
  2. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 7,916
  3. 2024GPT-4o System Cardcited by 4,467
  4. 2023Toolformer: Language Models Can Teach Themselves to Use Toolscited by 4,072
  5. 2018HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answeringcited by 3,686
  6. 2023Universal and Transferable Adversarial Attacks on Aligned Language Modelscited by 2,914
  7. 2023WebArena: A Realistic Web Environment for Building Autonomous Agentscited by 1,908
  8. 2023AutoGen: Enabling Next-Gen LLM Applications via Multi-Agent Conversationcited by 1,906
  9. 2023Not what you've signed up for: Compromising Real-World LLM-Integrated Applications with Indirect Prompt Injectioncited by 1,339
  10. 2023Jailbreaking Black Box Large Language Models in Twenty Queriescited by 1,326
  11. 2023Mind2Web: Towards a Generalist Agent for the Webcited by 1,210
  12. 2022WebShop: Towards Scalable Real-World Web Interaction with Grounded Language Agentscited by 1,197
27 more
  1. 2023Tree of Attacks: Jailbreaking Black-Box LLMs Automaticallycited by 619
  2. 2023API-Bank: A Comprehensive Benchmark for Tool-Augmented LLMscited by 574
  3. 2024GPT-4V(ision) is a Generalist Web Agent, if Groundedcited by 547
  4. 2024JailbreakBench: An Open Robustness Benchmark for Jailbreaking Large Language Modelscited by 494
  5. 2024The Instruction Hierarchy: Training LLMs to Prioritize Privileged Instructionscited by 441
  6. 2023SmoothLLM: Defending Large Language Models Against Jailbreaking Attackscited by 434
  7. 2024Great, Now Write an Article About That: The Crescendo Multi-Turn LLM Jailbreak Attackcited by 364
  8. 2023Formalizing and Benchmarking Prompt Injection Attacks and Defensescited by 328
  9. 2024StruQ: Defending Against Prompt Injection with Structured Queriescited by 325
  10. 2018Reinforcement Learning on Web Interfaces Using Workflow-Guided Explorationcited by 282
  11. 2023Certifying LLM Safety against Adversarial Promptingcited by 248
  12. 2024AutoDAN-Turbo: A Lifelong Agent for Strategy Self-Exploration to Jailbreak LLMscited by 185
  13. 2024SecAlign: Defending Against Prompt Injection with Preference Optimizationcited by 179
  14. 2024Assisting in Writing Wikipedia-like Articles From Scratch with Large Language Modelscited by 150
  15. 2025DataSentinel: A Game-Theoretic Detection of Prompt Injection Attackscited by 124
  16. 2023Jatmo: Prompt Injection Defense by Task-Specific Finetuningcited by 89
  17. 2024Agent-E: From Autonomous Web Navigation to Foundational Design Principles in Agentic Systemscited by 79
  18. 2024Instructional Segment Embedding: Improving LLM Safety with Instruction Hierarchycited by 62
  19. 2018Learning to Navigate the Webcited by 60
  20. 2024Bag of Tricks: Benchmarking of Jailbreak Attacks on LLMscited by 47
  21. 2025Defending Against Prompt Injection With a Few DefensiveTokenscited by 26
  22. 2024PAPILLON: Efficient and Stealthy Fuzz Testing-Powered Jailbreaks for LLMscited by 20
  23. 2025Can LLMs Help You at Work? A Sandbox for Evaluating LLM Agents in Enterprise Environmentscited by 15
  24. 2025TwinBreak: Jailbreaking LLM Security Alignments based on Twin Promptscited by 12
  25. 2016End-to-End Goal-Driven Web Navigationcited by 10
  26. 2024Self-interpreting Adversarial Imagescited by 7
  27. 2025Position: Certified Robustness Does Not (Yet) Imply Model Securitycited by 4

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 18.

18 papers.

Sort:

By area:

  1. 2026GPT-Red: Automated Red Teaming via Self-Play at Scalecited by 5
  2. 2026Agent Security is a Systems Problemcited by 2
  3. 2026Who Pays the Price? Stakeholder-Centric Prompt Injection Benchmarking for Real-world Web Agentscited by 2
  4. 2026Analyzing Defensive Misdirection Against Model-Guided Automated Attacks on Agentic AI Systemscited by 2
  5. 2026Agent Hacks Agent: Autoresearch for Production-Agent Red-Teamingcited by 2
  6. 2026Domain-Conditioned Safety in Frontier Computer-Using Agents: A 793-Episode Browser Benchmark, a Coding-Domain Cross-Reference, and a Reproducibility Audit of Recent Red-Teamingcited by 1
  7. 2026Prismata: Confining Cross-Site Prompt Injection in Web Agentscited by 1
  8. 2026Agent Against Agent: An Agentic System for Automatic Prompt Injection Red Teamingcited by 1
  9. 2026ToolHazard: Scaling Adversarial Environments for Security Evaluation and Alignment of LLM-based Agentscited by 1
  10. 2026Red-Teaming Claude Opus and ChatGPT-based Security Advisors for Trusted Execution Environmentscited by 0
  11. 2026Poisoned Playbooks: Demystifying Knowledge Poisoning Effects on AI Security Agentscited by 0
  12. 2026From Monoliths to Swarms: A Study of Attack Surface Evolution in the Transition to Multi-Agent Web Systemscited by 0
6 more
  1. 2026LoginTrap: Uncovering Task-Agnostic Phishing-Style Indirect Prompt Injection Attacks against LLM-based Web Agentscited by 0
  2. 2026Toward Metacognitive One-Shot Indirect Prompt Injection: Strategy Abstraction Via Outcome-Conditioned Reflectioncited by 0
  3. 2026Influence Is Not Authority: When Causal Guardrail Signals Make Legitimate Tool Use Look Like an Attack in Tool-Using LLM Agentscited by 0
  4. 2026SIR: Self-improving Red-teaming for Compute Use Agentscited by 0
  5. 2026Rethinking Indirect Prompt Injection as a Test-Time Search Problemcited by 0
  6. 2026Emergence World: Adversarial Stress-Testing of Long-Horizon Multi-Agent Systemscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025BrowseSafe: Understanding and Preventing Prompt Injection Within AI Browser Agents
  2. 2025Mind the Web: The Security of Web Use Agents
  3. 2026WebAgentGuard: A Reasoning-Driven Guard Model for Detecting Prompt Injection Attacks in Web Agents
  4. 2026WAAA! Web Adversaries Against Agentic Browsers
  5. 2026WARD: Adversarially Robust Defense of Web Agents Against Prompt Injections
  6. 2025WebGuard: Building a Generalizable Guardrail for Web Agents
  7. 2025The Hidden Dangers of Browsing AI Agents
  8. 2025It's a TRAP! Task-Redirecting Agent Persuasion Benchmark for Web Agents