StepJack: Benchmarking Computer-Use Agent Safety Against Multi-Step Indirect Prompt Injection
On the map: LLM agents and tool use → Agent threats, guardrails and governance
also in 1 more
Abstract (arXiv)
Computer-use agents (CUAs) face a growing threat from indirect prompt injection, where adversarial instructions are planted in the environment such as web pages. In this paper, we introduce multi-step indirect prompt injection, a new attack class against CUAs in which the adversarial goal is decomposed into multiple innocuous-looking sub-steps and distributed across a chain of pages referenced along the agent's navigation path. We develop a pipeline to automatically decompose an adversarial goal under the constraint that the execution of the decomposed sub-steps must achieve the original goal while optimizing the innocuousness of each decomposed sub-step. With this pipeline, we build StepJack, a CUA safety benchmark with 480 test examples. On this benchmark, we evaluate six state-of-the-art CUAs and find that at a fixed decomposition depth, multi-step attacks raise attack success rate (ASR) on three of six CUAs, by up to 31.2 points (e.g., GPT-5.4-mini: 41.7% at single-step to 72.9% at three-step); averaged over the five CUAs that can reliably follow the reference chain (all but EvoCUA-32B), ASR rises from 31.3% at single-step to 36.9% at three-step. Dataset and code are available at https://github.com/BorealisAI/StepJack.
Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Placed in Agent threats, guardrails and governance, one of 16 directions in LLM agents and tool use. Builds on 11 of its 24 references that are in the map. First posted on arXiv 6 Aug 2026.
Builds on
11 of 24 references.
Closest ancestors · 7
Same direction, newest first
- 2025PromptArmor: Simple yet Effective Prompt Injection Defenses21 Jul
- 2025OS-Harm: A Benchmark for Measuring Safety of Computer Use Agents17 Jun
- 2025VPI-Bench: Visual Prompt Injection Attacks for Computer-Use Agents3 Jun
- 2025RedTeamCUA: Realistic Adversarial Testing of Computer-Use Agents in Hybrid Web-OS Environments28 May
- 2025WASP: Benchmarking Web Agent Security Against Prompt Injection Attacks22 Apr
- 2025DoomArena: A framework for Testing AI Agents Against Evolving Security Threats18 Apr
- 2025SafeArena: Evaluating the Safety of Autonomous Web Agents6 Mar
Other references · 4
- 2023Not what you've signed up for: Compromising Real-World LLM-Integrated Applications with Indirect Prompt Injectioncited by 1,339
- 2024OSWorld: Benchmarking Multimodal Agents for Open-Ended Tasks in Real Computer Environmentscited by 1,134
- 2024Great, Now Write an Article About That: The Crescendo Multi-Turn LLM Jailbreak Attackcited by 364
- 2026EvoCUA: Evolving Computer Use Agents via Learning from Scalable Synthetic Experiencecited by 51
Built on by
Citing papers per year: 2026: 2.
2 papers.
Similar, not cited
Close in wording, no citation either way
- 2026Invisible Ink Threats: Adversarial Goals Behind Legitimate Tasks in Computer-Use Agents
- 2025A Systematization of Security Vulnerabilities in Computer Use Agents
- 2026Domain-Conditioned Safety in Frontier Computer-Using Agents: A 793-Episode Browser Benchmark, a Coding-Domain Cross-Reference, and a Reproducibility Audit of Recent Red-Teaming
- 2026Defense Against Indirect Prompt Injection via Tool Result Parsing
- 2026Assessing Automated Prompt Injection Attacks in Agentic Environments
- 2026The Blind Spot of Agent Safety: How Benign User Instructions Expose Critical Vulnerabilities in Computer-Use Agents
- 2026MIRAGE: Stealthy Visual Prompt Injection for Vulnerability Detection in Web Agents
- 2026Silent Egress: When Implicit Prompt Injection Makes LLM Agents Leak Without a Trace