MUZZLE: Adaptive Agentic Red-Teaming of Web Agents Against Indirect Prompt Injection Attacks
On the map: LLM agents and tool use → Agent threats, guardrails and governance
also in 1 more
Abstract (arXiv)
Large language model (LLM) based web agents are increasingly deployed to automate complex online tasks by directly interacting with web sites and performing actions on users' behalf. While these agents offer powerful capabilities, their design exposes them to indirect prompt injection attacks embedded in untrusted web content, enabling adversaries to hijack agent behavior and violate user intent. Despite growing awareness of this threat, existing evaluations rely on fixed attack templates, manually selected injection surfaces, or narrowly scoped scenarios, limiting their ability to capture realistic, adaptive attacks encountered in practice. We present MUZZLE, an automated agentic framework for evaluating the security of web agents against indirect prompt injection attacks. MUZZLE utilizes the agent's trajectories to automatically identify high-salience injection surfaces, and adaptively generate context-aware malicious instructions that target violations of confidentiality, integrity, and availability. Unlike prior approaches, MUZZLE adapts its attack strategy based on the agent's observed execution trajectory and iteratively refines attacks using feedback from failed executions. We evaluate MUZZLE across diverse web applications, user tasks, and agent configurations, demonstrating its ability to automatically and adaptively assess the security of web agents with minimal human intervention. Our results show that MUZZLE effectively discovers 44 new attacks on 4 web applications with 10 adversarial objectives that violate confidentiality, availability, or privacy properties across different LLMs and agent scaffolds. MUZZLE also identifies novel attack strategies, including 3 cross-application prompt injection attacks and an agent-tailored phishing scenario.
Key facts. Cited by 18 papers in the map (data through 7 Oct 2026). Placed in Agent threats, guardrails and governance, one of 16 directions in LLM agents and tool use. Builds on 50 of its 89 references that are in the map. First posted on arXiv 9 Feb 2026.
Builds on
50 of 89 references.
Closest ancestors · 11
Same direction, newest first
- 2025RedCodeAgent: Automatic Red-teaming Agent against Diverse Code Agents2 Oct
- 2025OS-Harm: A Benchmark for Measuring Safety of Computer Use Agents17 Jun
- 2025AgentVigil: Generic Black-Box Red-teaming for Indirect Prompt Injection against LLM Agents9 May
- 2025WASP: Benchmarking Web Agent Security Against Prompt Injection Attacks22 Apr
- 2025MELON: Provable Defense Against Indirect Prompt Injection Attacks in AI Agents7 Feb
- 2024AdvAgent: Controllable Blackbox Red-teaming on Web Agents22 Oct
- 2024AgentHarm: A Benchmark for Measuring Harmfulness of LLM Agents11 Oct
- 2024Agent Security Bench (ASB): Formalizing and Benchmarking Attacks and Defenses in LLM-based Agents3 Oct
- 2024AgentPoison: Red-teaming LLM Agents via Poisoning Memory or Knowledge Bases17 Jul
- 2024Dissecting Adversarial Robustness of Multimodal LM Agents18 Jun
- 2024InjecAgent: Benchmarking Indirect Prompt Injections in Tool-Integrated Large Language Model Agents5 Mar
Other references · 39
- 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,355
- 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 7,916
- 2024GPT-4o System Cardcited by 4,467
- 2023Toolformer: Language Models Can Teach Themselves to Use Toolscited by 4,072
- 2018HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answeringcited by 3,686
- 2023Universal and Transferable Adversarial Attacks on Aligned Language Modelscited by 2,914
- 2023WebArena: A Realistic Web Environment for Building Autonomous Agentscited by 1,908
- 2023AutoGen: Enabling Next-Gen LLM Applications via Multi-Agent Conversationcited by 1,906
- 2023Not what you've signed up for: Compromising Real-World LLM-Integrated Applications with Indirect Prompt Injectioncited by 1,339
- 2023Jailbreaking Black Box Large Language Models in Twenty Queriescited by 1,326
- 2023Mind2Web: Towards a Generalist Agent for the Webcited by 1,210
- 2022WebShop: Towards Scalable Real-World Web Interaction with Grounded Language Agentscited by 1,197
27 more
- 2023Tree of Attacks: Jailbreaking Black-Box LLMs Automaticallycited by 619
- 2023API-Bank: A Comprehensive Benchmark for Tool-Augmented LLMscited by 574
- 2024GPT-4V(ision) is a Generalist Web Agent, if Groundedcited by 547
- 2024JailbreakBench: An Open Robustness Benchmark for Jailbreaking Large Language Modelscited by 494
- 2024The Instruction Hierarchy: Training LLMs to Prioritize Privileged Instructionscited by 441
- 2023SmoothLLM: Defending Large Language Models Against Jailbreaking Attackscited by 434
- 2024Great, Now Write an Article About That: The Crescendo Multi-Turn LLM Jailbreak Attackcited by 364
- 2023Formalizing and Benchmarking Prompt Injection Attacks and Defensescited by 328
- 2024StruQ: Defending Against Prompt Injection with Structured Queriescited by 325
- 2018Reinforcement Learning on Web Interfaces Using Workflow-Guided Explorationcited by 282
- 2023Certifying LLM Safety against Adversarial Promptingcited by 248
- 2024AutoDAN-Turbo: A Lifelong Agent for Strategy Self-Exploration to Jailbreak LLMscited by 185
- 2024SecAlign: Defending Against Prompt Injection with Preference Optimizationcited by 179
- 2024Assisting in Writing Wikipedia-like Articles From Scratch with Large Language Modelscited by 150
- 2025DataSentinel: A Game-Theoretic Detection of Prompt Injection Attackscited by 124
- 2023Jatmo: Prompt Injection Defense by Task-Specific Finetuningcited by 89
- 2024Agent-E: From Autonomous Web Navigation to Foundational Design Principles in Agentic Systemscited by 79
- 2024Instructional Segment Embedding: Improving LLM Safety with Instruction Hierarchycited by 62
- 2018Learning to Navigate the Webcited by 60
- 2024Bag of Tricks: Benchmarking of Jailbreak Attacks on LLMscited by 47
- 2025Defending Against Prompt Injection With a Few DefensiveTokenscited by 26
- 2024PAPILLON: Efficient and Stealthy Fuzz Testing-Powered Jailbreaks for LLMscited by 20
- 2025Can LLMs Help You at Work? A Sandbox for Evaluating LLM Agents in Enterprise Environmentscited by 15
- 2025TwinBreak: Jailbreaking LLM Security Alignments based on Twin Promptscited by 12
- 2016End-to-End Goal-Driven Web Navigationcited by 10
- 2024Self-interpreting Adversarial Imagescited by 7
- 2025Position: Certified Robustness Does Not (Yet) Imply Model Securitycited by 4
Built on by
Citing papers per year: 2026: 18.
18 papers.
Sort:
By area:
- 2026GPT-Red: Automated Red Teaming via Self-Play at Scalecited by 5
- 2026Agent Security is a Systems Problemcited by 2
- 2026Who Pays the Price? Stakeholder-Centric Prompt Injection Benchmarking for Real-world Web Agentscited by 2
- 2026Analyzing Defensive Misdirection Against Model-Guided Automated Attacks on Agentic AI Systemscited by 2
- 2026Agent Hacks Agent: Autoresearch for Production-Agent Red-Teamingcited by 2
- 2026Domain-Conditioned Safety in Frontier Computer-Using Agents: A 793-Episode Browser Benchmark, a Coding-Domain Cross-Reference, and a Reproducibility Audit of Recent Red-Teamingcited by 1
- 2026Prismata: Confining Cross-Site Prompt Injection in Web Agentscited by 1
- 2026Agent Against Agent: An Agentic System for Automatic Prompt Injection Red Teamingcited by 1
- 2026ToolHazard: Scaling Adversarial Environments for Security Evaluation and Alignment of LLM-based Agentscited by 1
- 2026Red-Teaming Claude Opus and ChatGPT-based Security Advisors for Trusted Execution Environmentscited by 0
- 2026Poisoned Playbooks: Demystifying Knowledge Poisoning Effects on AI Security Agentscited by 0
- 2026From Monoliths to Swarms: A Study of Attack Surface Evolution in the Transition to Multi-Agent Web Systemscited by 0
6 more
- 2026LoginTrap: Uncovering Task-Agnostic Phishing-Style Indirect Prompt Injection Attacks against LLM-based Web Agentscited by 0
- 2026Toward Metacognitive One-Shot Indirect Prompt Injection: Strategy Abstraction Via Outcome-Conditioned Reflectioncited by 0
- 2026Influence Is Not Authority: When Causal Guardrail Signals Make Legitimate Tool Use Look Like an Attack in Tool-Using LLM Agentscited by 0
- 2026SIR: Self-improving Red-teaming for Compute Use Agentscited by 0
- 2026Rethinking Indirect Prompt Injection as a Test-Time Search Problemcited by 0
- 2026Emergence World: Adversarial Stress-Testing of Long-Horizon Multi-Agent Systemscited by 0
- 2026Emergence World: Adversarial Stress-Testing of Long-Horizon Multi-Agent Systems15 Sep 2026
- 2026Rethinking Indirect Prompt Injection as a Test-Time Search Problem3 Sep 2026
- 2026SIR: Self-improving Red-teaming for Compute Use Agents31 Aug 2026
- 2026Influence Is Not Authority: When Causal Guardrail Signals Make Legitimate Tool Use Look Like an Attack in Tool-Using LLM Agents30 Aug 2026
- 2026ToolHazard: Scaling Adversarial Environments for Security Evaluation and Alignment of LLM-based Agents12 Aug 2026
- 2026Toward Metacognitive One-Shot Indirect Prompt Injection: Strategy Abstraction Via Outcome-Conditioned Reflection9 Aug 2026
- 2026Agent Against Agent: An Agentic System for Automatic Prompt Injection Red Teaming5 Aug 2026
- 2026LoginTrap: Uncovering Task-Agnostic Phishing-Style Indirect Prompt Injection Attacks against LLM-based Web Agents5 Aug 2026
- 2026From Monoliths to Swarms: A Study of Attack Surface Evolution in the Transition to Multi-Agent Web Systems31 Jul 2026
- 2026GPT-Red: Automated Red Teaming via Self-Play at Scale28 Jul 2026
- 2026Agent Hacks Agent: Autoresearch for Production-Agent Red-Teaming13 Jul 2026
- 2026Prismata: Confining Cross-Site Prompt Injection in Web Agents9 Jul 2026
6 more
- 2026Poisoned Playbooks: Demystifying Knowledge Poisoning Effects on AI Security Agents23 Jun 2026
- 2026Analyzing Defensive Misdirection Against Model-Guided Automated Attacks on Agentic AI Systems18 Jun 2026
- 2026Who Pays the Price? Stakeholder-Centric Prompt Injection Benchmarking for Real-world Web Agents11 Jun 2026
- 2026Domain-Conditioned Safety in Frontier Computer-Using Agents: A 793-Episode Browser Benchmark, a Coding-Domain Cross-Reference, and a Reproducibility Audit of Recent Red-Teaming3 Jun 2026
- 2026Agent Security is a Systems Problem18 May 2026
- 2026Red-Teaming Claude Opus and ChatGPT-based Security Advisors for Trusted Execution Environments23 Feb 2026
- 2026Who Pays the Price? Stakeholder-Centric Prompt Injection Benchmarking for Real-world Web Agentscited by 2
- 2026Analyzing Defensive Misdirection Against Model-Guided Automated Attacks on Agentic AI Systemscited by 2
- 2026Agent Hacks Agent: Autoresearch for Production-Agent Red-Teamingcited by 2
- 2026Domain-Conditioned Safety in Frontier Computer-Using Agents: A 793-Episode Browser Benchmark, a Coding-Domain Cross-Reference, and a Reproducibility Audit of Recent Red-Teamingcited by 1
- 2026Prismata: Confining Cross-Site Prompt Injection in Web Agentscited by 1
- 2026Agent Against Agent: An Agentic System for Automatic Prompt Injection Red Teamingcited by 1
- 2026ToolHazard: Scaling Adversarial Environments for Security Evaluation and Alignment of LLM-based Agentscited by 1
- 2026From Monoliths to Swarms: A Study of Attack Surface Evolution in the Transition to Multi-Agent Web Systemscited by 0
- 2026LoginTrap: Uncovering Task-Agnostic Phishing-Style Indirect Prompt Injection Attacks against LLM-based Web Agentscited by 0
- 2026Influence Is Not Authority: When Causal Guardrail Signals Make Legitimate Tool Use Look Like an Attack in Tool-Using LLM Agentscited by 0
- 2026SIR: Self-improving Red-teaming for Compute Use Agentscited by 0
- 2026Emergence World: Adversarial Stress-Testing of Long-Horizon Multi-Agent Systemscited by 0
- 2026Emergence World: Adversarial Stress-Testing of Long-Horizon Multi-Agent Systems15 Sep 2026
- 2026SIR: Self-improving Red-teaming for Compute Use Agents31 Aug 2026
- 2026Influence Is Not Authority: When Causal Guardrail Signals Make Legitimate Tool Use Look Like an Attack in Tool-Using LLM Agents30 Aug 2026
- 2026ToolHazard: Scaling Adversarial Environments for Security Evaluation and Alignment of LLM-based Agents12 Aug 2026
- 2026Agent Against Agent: An Agentic System for Automatic Prompt Injection Red Teaming5 Aug 2026
- 2026LoginTrap: Uncovering Task-Agnostic Phishing-Style Indirect Prompt Injection Attacks against LLM-based Web Agents5 Aug 2026
- 2026From Monoliths to Swarms: A Study of Attack Surface Evolution in the Transition to Multi-Agent Web Systems31 Jul 2026
- 2026Agent Hacks Agent: Autoresearch for Production-Agent Red-Teaming13 Jul 2026
- 2026Prismata: Confining Cross-Site Prompt Injection in Web Agents9 Jul 2026
- 2026Analyzing Defensive Misdirection Against Model-Guided Automated Attacks on Agentic AI Systems18 Jun 2026
- 2026Who Pays the Price? Stakeholder-Centric Prompt Injection Benchmarking for Real-world Web Agents11 Jun 2026
- 2026Domain-Conditioned Safety in Frontier Computer-Using Agents: A 793-Episode Browser Benchmark, a Coding-Domain Cross-Reference, and a Reproducibility Audit of Recent Red-Teaming3 Jun 2026
- 2026Red-Teaming Claude Opus and ChatGPT-based Security Advisors for Trusted Execution Environmentscited by 0
- 2026Toward Metacognitive One-Shot Indirect Prompt Injection: Strategy Abstraction Via Outcome-Conditioned Reflectioncited by 0
- 2026Rethinking Indirect Prompt Injection as a Test-Time Search Problemcited by 0
- 2026Rethinking Indirect Prompt Injection as a Test-Time Search Problem3 Sep 2026
- 2026Toward Metacognitive One-Shot Indirect Prompt Injection: Strategy Abstraction Via Outcome-Conditioned Reflection9 Aug 2026
- 2026Red-Teaming Claude Opus and ChatGPT-based Security Advisors for Trusted Execution Environments23 Feb 2026
- 2026GPT-Red: Automated Red Teaming via Self-Play at Scalecited by 5
- 2026GPT-Red: Automated Red Teaming via Self-Play at Scale28 Jul 2026
- 2026Agent Security is a Systems Problemcited by 2
- 2026Agent Security is a Systems Problem18 May 2026
Similar, not cited
Close in wording, no citation either way
- 2025BrowseSafe: Understanding and Preventing Prompt Injection Within AI Browser Agents
- 2025Mind the Web: The Security of Web Use Agents
- 2026WebAgentGuard: A Reasoning-Driven Guard Model for Detecting Prompt Injection Attacks in Web Agents
- 2026WAAA! Web Adversaries Against Agentic Browsers
- 2026WARD: Adversarially Robust Defense of Web Agents Against Prompt Injections
- 2025WebGuard: Building a Generalizable Guardrail for Web Agents
- 2025The Hidden Dangers of Browsing AI Agents
- 2025It's a TRAP! Task-Redirecting Agent Persuasion Benchmark for Web Agents