mlsanityData to 7 Oct 2026

Attacking Vision-Language Computer Agents via Pop-ups

arXiv 2411.02391 ↗First posted 4 Nov 2024cs.CLcited by 112

On the map: LLM agents and tool use → Agent threats, guardrails and governance

Abstract (arXiv)

Autonomous agents powered by large vision and language models (VLM) have demonstrated significant potential in completing daily computer tasks, such as browsing the web to book travel and operating desktop software, which requires agents to understand these interfaces. Despite such visual inputs becoming more integrated into agentic applications, what types of risks and attacks exist around them still remain unclear. In this work, we demonstrate that VLM agents can be easily attacked by a set of carefully designed adversarial pop-ups, which human users would typically recognize and ignore. This distraction leads agents to click these pop-ups instead of performing their tasks as usual. Integrating these pop-ups into existing agent testing environments like OSWorld and VisualWebArena leads to an attack success rate (the frequency of the agent clicking the pop-ups) of 86% on average and decreases the task success rate by 47%. Basic defense techniques, such as asking the agent to ignore pop-ups or including an advertisement notice, are ineffective against the attack.

Key facts. Cited by 112 papers in the map (data through 7 Oct 2026). Placed in Agent threats, guardrails and governance, one of 16 directions in LLM agents and tool use. Builds on 24 of its 42 references that are in the map. First posted on arXiv 4 Nov 2024.

Builds on

24 of 42 references.

Closest ancestors · 5

Same direction, newest first

  1. 2024AdvAgent: Controllable Blackbox Red-teaming on Web Agents22 Oct
  2. 2024EIA: Environmental Injection Attack on Generalist Web Agents for Privacy Leakage17 Sep
  3. 2024InjecAgent: Benchmarking Indirect Prompt Injections in Tool-Integrated Large Language Model Agents5 Mar
  4. 2024Watch Out for Your Agents! Investigating Backdoor Threats to LLM-Based Agents17 Feb
  5. 2023Identifying the Risks of LM Agents with an LM-Emulated Sandbox25 Sep

Other references · 19

  1. 2023GPT-4 Technical Reportcited by 18,132
  2. 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,355
  3. 2024Gemini 1.5: Unlocking multimodal understanding across millions of tokens of contextcited by 3,175
  4. 2023Universal and Transferable Adversarial Attacks on Aligned Language Modelscited by 2,914
  5. 2023MMMU: A Massive Multi-discipline Multimodal Understanding and Reasoning Benchmark for Expert AGIcited by 2,044
  6. 2023WebArena: A Realistic Web Environment for Building Autonomous Agentscited by 1,908
  7. 2023MathVista: Evaluating Mathematical Reasoning of Foundation Models in Visual Contextscited by 1,582
  8. 2024τ-bench: A Benchmark for Tool-Agent-User Interaction in Real-World Domainscited by 1,280
  9. 2023Mind2Web: Towards a Generalist Agent for the Webcited by 1,210
  10. 2024OSWorld: Benchmarking Multimodal Agents for Open-Ended Tasks in Real Computer Environmentscited by 1,134
  11. 2024SeeClick: Harnessing GUI Grounding for Advanced Visual GUI Agentscited by 553
  12. 2024GPT-4V(ision) is a Generalist Web Agent, if Groundedcited by 547
7 more
  1. 2024How Johnny Can Persuade LLMs to Jailbreak Them: Rethinking Persuasion to Challenge AI Safety by Humanizing LLMscited by 533
  2. 2024The Instruction Hierarchy: Training LLMs to Prioritize Privileged Instructionscited by 441
  3. 2023Paraphrasing evades detectors of AI-generated text, but retrieval is an effective defensecited by 401
  4. 2023Set-of-Mark Prompting Unleashes Extraordinary Visual Grounding in GPT-4Vcited by 291
  5. 2024Agent S: An Open Agentic Framework that Uses Computers Like a Humancited by 182
  6. 2024Caution for the Environment: Multimodal LLM Agents are Susceptible to Environmental Distractionscited by 55
  7. 2024Can LLMs be Fooled? Investigating Vulnerabilities in LLMscited by 5

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 2; 2025: 67; 2026: 43.

112 papers.

Sort:

By area:

  1. 2024Large Language Model-Brained GUI Agents: A Surveycited by 160
  2. 2025ShieldAgent: Shielding Agents via Verifiable Safety Policy Reasoningcited by 144
  3. 2025WASP: Benchmarking Web Agent Security Against Prompt Injection Attackscited by 140
  4. 2025OS-Harm: A Benchmark for Measuring Safety of Computer Use Agentscited by 104
  5. 2025OpenAgentSafety: A Comprehensive Framework for Evaluating Real-World AI Agent Safetycited by 71
  6. 2025MELON: Provable Defense Against Indirect Prompt Injection Attacks in AI Agentscited by 67
  7. 2025Safety at Scale: A Comprehensive Survey of Large Model and Agent Safetycited by 63
  8. 2025DRIFT: Dynamic Rule-Based Defense with Injection Isolation for Securing LLM Agentscited by 59
  9. 2025The Obvious Invisible Threat: LLM-Powered GUI Agents' Vulnerability to Fine-Print Injectionscited by 55
  10. 2025WebInject: Prompt Injection Attack to Web Agentscited by 48
  11. 2025Creating General User Models from Computer Usecited by 44
  12. 2025AgentVigil: Generic Black-Box Red-teaming for Indirect Prompt Injection against LLM Agentscited by 42
38 more
  1. 2025RedTeamCUA: Realistic Adversarial Testing of Computer-Use Agents in Hybrid Web-OS Environmentscited by 42
  2. 2025VPI-Bench: Visual Prompt Injection Attacks for Computer-Use Agentscited by 42
  3. 2025OS Agents: A Survey on MLLM-based Agents for General Computing Devices Usecited by 35
  4. 2025RiOSWorld: Benchmarking the Risk of Multimodal Computer-Use Agentscited by 34
  5. 2025Towards Trustworthy GUI Agents: A Surveycited by 33
  6. 2024AutoDroid-V2: Boosting SLM-based GUI Agents via Code Generationcited by 31
  7. 2025DoomArena: A framework for Testing AI Agents Against Evolving Security Threatscited by 31
  8. 2025macOSWorld: A Multilingual Interactive Benchmark for GUI Agentscited by 31
  9. 2025API Agents vs. GUI Agents: Divergence and Convergencecited by 28
  10. 2025Why Are Web AI Agents More Vulnerable Than Standalone LLMs? A Security Analysiscited by 27
  11. 2025Agentic AI Security: Threats, Defenses, Evaluation, and Open Challengescited by 26
  12. 2025AgentSentinel: An End-to-End and Real-Time Security Defense Framework for Computer-Use Agentscited by 25
  13. 2026"Humans welcome to observe": A First Look at the Agent Social Network Moltbookcited by 25
  14. 2025MLA-Trust: Benchmarking Trustworthiness of Multimodal LLM Agents in GUI Environmentscited by 23
  15. 2025BrowseSafe: Understanding and Preventing Prompt Injection Within AI Browser Agentscited by 23
  16. 2025Evaluating the Robustness of Multimodal Agents Against Active Environmental Injection Attackscited by 21
  17. 2025WAInjectBench: Benchmarking Prompt Injection Detections for Web Agentscited by 21
  18. 2025WebGuard: Building a Generalizable Guardrail for Web Agentscited by 20
  19. 2025Mirage-1: Augmenting and Updating GUI Agent with Hierarchical Multimodal Skillscited by 18
  20. 2025SecureWebArena: A Holistic Security Evaluation Benchmark for LVLM-based Web Agentscited by 17
  21. 2025Test-Time Adaptation for LLM Agents via Environment Interactioncited by 17
  22. 2025MIRAGE-Bench: LLM Agent is Hallucinating and Where to Find Themcited by 16
  23. 2025FocusAgent: Simple Yet Effective Ways of Trimming the Large Context of Web Agentscited by 16
  24. 2025Dark Patterns Meet GUI Agents: LLM Agent Susceptibility to Manipulative Interfaces and the Role of Human Oversightcited by 15
  25. 2026CaMeLs Can Use Computers Too: System-level Security for Computer Use Agentscited by 15
  26. 2025On the Robustness of GUI Grounding Models Against Image Attackscited by 14
  27. 2025Think Twice Before You Act: Enhancing Agent Behavioral Safety with Thought Correctioncited by 14
  28. 2025Investigating the Impact of Dark Patterns on LLM-Based Web Agentscited by 13
  29. 2025DECEPTICON: How Dark Patterns Manipulate Web Agentscited by 13
  30. 2025MIP against Agent: Malicious Image Patches Hijacking Multimodal OS Agentscited by 12
  31. 2025Exploring Typographic Visual Prompts Injection Threats in Cross-Modality Generation Modelscited by 12
  32. 2025GhostEI-Bench: Do Mobile Agents Resilience to Environmental Injection in Dynamic On-Device Environments?cited by 12
  33. 2025Toward a Human-Centered Evaluation Framework for Trustworthy LLM-Powered GUI Agentscited by 11
  34. 2025HarmonyGuard: Toward Safety and Utility in Web Agents via Adaptive Policy Enhancement and Dual-Objective Optimizationcited by 10
  35. 2025A Framework for Studying AI Agent Behavior: Evidence from Consumer Choice Experimentscited by 10
  36. 2025OpenApps: Simulating Environment Variations to Measure UI-Agent Reliabilitycited by 10
  37. 2026WebSentinel: Detecting and Localizing Prompt Injection Attacks for Web Agentscited by 10
  38. 2025AdInject: Real-World Black-Box Attacks on Web Agents via Advertising Deliverycited by 9

50 most cited of 112 All 112 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2024A Trembling House of Cards? Mapping Adversarial Attacks against Language Agents
  2. 2026Adversarial attacks against Modern Vision-Language Models
  3. 2025AttackVLA: Benchmarking Adversarial and Backdoor Attacks on Vision-Language-Action Models
  4. 2025Invisible to Humans, Triggered by Agents: Stealthy Jailbreak Attacks on Mobile Vision-Language Agents
  5. 2026SABER: A Stealthy Agentic Black-Box Attack Framework for Vision-Language-Action Models
  6. 2025When Data Manipulation Meets Attack Goals: An In-depth Survey of Attacks for VLMs
  7. 2025ADVEDM:Fine-grained Adversarial Attack against VLM-based Embodied Agents
  8. 2026Hijacking Robots with a Piece of Paper: A Systematic Study of Physical Prompt Injection in VLM-Controlled Robots