Securing Computer-Use Agents: A Unified Architecture-Lifecycle Framework for Deployment-Grounded Reliability
On the map: LLM agents and tool use → Agent threats, guardrails and governance
Abstract (arXiv)
Computer-use agents(CUAs)are moving frombounded benchmarks toward real software environments, wherethey operate browsers, desktops, mobile applications, flesystems,terminals, and tool backends. In such settings, reliability isno longer captured by task success alone: perception errors,planning drift, memory use, tool mediation, permission scope,and runtime oversight jointly determine whether agent actionsremain aligned with user intent, Existing surveys organize theCUA landscape by methods, platforms, benchmarks, or securitythreats, but less explicitly connect capability formation, author-ity exposure, failure manifestation, and control placement. Toaddress this gap, the article develops an architecture-lifecycleframework for deployment-grounded reliability in CUAs. Thearchitectural view analyzes Perception, Decision, and Executionas coupled layers that transform software observations intoauthority-bearing actions, The lifecycle view examines Creation.Deployment, Operation, and Maintenance as stages in which priorsare learned, tools and permissions are bound, runtime trajecto.ries are stressed, and assurance must be preserved under drift.Using this lens, the analysis synthesizes representative systems,benchmarks, and security/privacy studies; distinguishes wherefailures become visible from where their enabling conditions areintroduced, and maps recurring intervention surfaces for controloversight, and assurance. OpenClaw is used only as a public moti.vating example of an open deployment pattern, not as a verifedinternal case study. The conclusion highlights open challengesin controllable grounding, long-horizon constraint preservation,safe authority binding, mixed-trust runtime defense, privacy-preserving memory,and continual assurance.
Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Agent threats, guardrails and governance, one of 16 directions in LLM agents and tool use. Builds on 175 of its 191 references that are in the map. First posted on arXiv 8 May 2026.
Builds on
175 of 191 references.
Closest ancestors · 56
Same direction, newest first
- 2026The Blind Spot of Agent Safety: How Benign User Instructions Expose Critical Vulnerabilities in Computer-Use Agents12 Apr
- 2026Preference Redirection via Attention Concentration: An Attack on Computer Use Agents9 Apr
- 2026Are GUI Agents Focused Enough? Automated Distraction via Semantic-level UI Element Injection9 Apr
- 2026WebSP-Eval: Evaluating Web Agents on Website Security and Privacy Tasks7 Apr
- 2026Your Agent, Their Asset: A Real-World Safety Analysis of OpenClaw6 Apr
- 2026AgentRAE: Remote Action Execution through Notification-based Visual Backdoors against Screenshots-based Mobile GUI Agents24 Mar
- 2026SlowBA: An efficiency backdoor attack towards VLM-based GUI agents9 Mar
- 2026Dual-Modality Multi-Stage Adversarial Safety Training: Robustifying Multimodal Web Agents Against Cross-Modal Attacks4 Mar
- 2026When Actions Go Off-Task: Detecting and Correcting Misaligned Actions in Computer-Use Agents9 Feb
- 2026From Assistant to Double Agent: Formalizing and Benchmarking Attacks on OpenClaw for Personalized Local AI Agent9 Feb
- 2026When Benign Inputs Lead to Severe Harms: Eliciting Unsafe Unintended Behaviors of Computer-Use Agents9 Feb
- 2026WebSentinel: Detecting and Localizing Prompt Injection Attacks for Web Agents3 Feb
38 more
- 2026LPS-Bench: Benchmarking Safety Awareness of Computer-Use Agents in Long-Horizon Planning under Benign and Adversarial Scenarios3 Feb
- 2026Risky-Bench: Probing Agentic Safety Risks under Real-World Deployment3 Feb
- 2026AgentDoG: A Diagnostic Guardrail Framework for AI Agent Safety and Security26 Jan
- 2026MirrorGuard: Toward Secure Computer-Use Agents via Simulation-to-Real Reasoning Correction19 Jan
- 2026Mind the Gap: Action Rebinding Attacks against Android GUI Agents18 Jan
- 2026CaMeLs Can Use Computers Too: System-level Security for Computer Use Agents14 Jan
- 2026A Survey of Agentic AI and Cybersecurity: Challenges, Opportunities and Use-case Prototypes8 Jan
- 2025It's a TRAP! Task-Redirecting Agent Persuasion Benchmark for Web Agents29 Dec
- 2025DECEPTICON: How Dark Patterns Manipulate Web Agents28 Dec
- 2025Mitigating Indirect Prompt Injection via Instruction-Following Intent Analysis30 Nov
- 2025PropensityBench: Evaluating Latent Safety Risks in Large Language Models via an Agentic Approach24 Nov
- 2025GhostEI-Bench: Do Mobile Agents Resilience to Environmental Injection in Dynamic On-Device Environments?23 Oct
- 2025Investigating the Impact of Dark Patterns on LLM-Based Web Agents20 Oct
- 2025MCP Security Bench (MSB): Benchmarking Attacks Against Model Context Protocol in LLM Agents14 Oct
- 2025HackWorld: Evaluating Computer-Use Agents on Exploiting Web Application Vulnerabilities14 Oct
- 2025VeriGuard: Enhancing LLM Agent Safety via Verified Code Generation3 Oct
- 2025WAInjectBench: Benchmarking Prompt Injection Detections for Web Agents1 Oct
- 2025Secure and Efficient Access Control for Computer-Use Agents via Context Space26 Sep
- 2025Privacy in Action: Towards Realistic Privacy Mitigation and Evaluation for LLM-Powered Agents22 Sep
- 2025Environmental Injection Attacks against GUI Agents in Realistic Dynamic Environments14 Sep
- 2025AgentSentinel: An End-to-End and Real-Time Security Defense Framework for Computer-Use Agents9 Sep
- 2025Measuring Harmfulness of Computer-Using Agents31 Jul
- 2025A Systematization of Security Vulnerabilities in Computer Use Agents7 Jul
- 2025OS-Harm: A Benchmark for Measuring Safety of Computer Use Agents17 Jun
- 2025VPI-Bench: Visual Prompt Injection Attacks for Computer-Use Agents3 Jun
- 2025RiOSWorld: Benchmarking the Risk of Multimodal Computer-Use Agents31 May
- 2025RedTeamCUA: Realistic Adversarial Testing of Computer-Use Agents in Hybrid Web-OS Environments28 May
- 2025Hidden Ghost Hand: Unveiling Backdoor Vulnerabilities in MLLM-Powered Mobile GUI Agents20 May
- 2025WebInject: Prompt Injection Attack to Web Agents16 May
- 2025A Survey on the Safety and Security Threats of Computer-Using Agents: JARVIS or Ultron?16 May
- 2025WASP: Benchmarking Web Agent Security Against Prompt Injection Attacks22 Apr
- 2025Les Dissonances: Cross-Tool Harvesting and Polluting in Pool-of-Tools Empowered LLM Agents4 Apr
- 2025ShieldAgent: Shielding Agents via Verifiable Safety Policy Reasoning26 Mar
- 2025MIP against Agent: Malicious Image Patches Hijacking Multimodal OS Agents13 Mar
- 2025AgentDAM: Privacy Leakage Evaluation for Autonomous Web Agents12 Mar
- 2025In-Context Defense in Computer Agents: An Empirical Study12 Mar
- 2025Evaluating the Robustness of Multimodal Agents Against Active Environmental Injection Attacks18 Feb
- 2024Prompt Infection: LLM-to-LLM Prompt Injection within Multi-Agent Systems9 Oct
50 newest of 56 All 56 (JSON)
Other references · 119
- 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,355
- 2023Reflexion: Language Agents with Verbal Reinforcement Learningcited by 4,064
- 2023Tree of Thoughts: Deliberate Problem Solving with Large Language Modelscited by 3,155
- 2023Qwen-VL: A Versatile Vision-Language Model for Understanding, Localization, Text Reading, and Beyondcited by 2,179
- 2023WebArena: A Realistic Web Environment for Building Autonomous Agentscited by 1,908
- 2023Mind2Web: Towards a Generalist Agent for the Webcited by 1,210
- 2024OSWorld: Benchmarking Multimodal Agents for Open-Ended Tasks in Real Computer Environmentscited by 1,134
- 2023CogAgent: A Visual Language Model for GUI Agentscited by 692
- 2024SeeClick: Harnessing GUI Grounding for Advanced Visual GUI Agentscited by 553
- 2024WebVoyager: Building an End-to-End Web Agent with Large Multimodal Modelscited by 424
- 2023AppAgent: Multimodal Agents as Smartphone Userscited by 407
- 2023A Real-World WebAgent with Planning, Long Context Understanding, and Program Synthesiscited by 350
38 more
- 2024WorkArena: How Capable Are Web Agents at Solving Common Knowledge Work Tasks?cited by 348
- 2023AgentTuning: Enabling Generalized Agent Abilities for LLMscited by 319
- 2023Android in the Wild: A Large-Scale Dataset for Android Device Controlcited by 314
- 2024Mobile-Agent: Autonomous Multi-Modal Mobile Device Agent with Visual Perceptioncited by 283
- 2025ScreenSpot-Pro: GUI Grounding for Professional High-Resolution Computer Usecited by 246
- 2025GUI-R1 : A Generalist R1-Style Vision-Language Action Model For GUI Agentscited by 240
- 2025UI-TARS-2 Technical Report: Advancing GUI Agent with Multi-Turn Reinforcement Learningcited by 205
- 2024OS-Copilot: Towards Generalist Computer Agents with Self-Improvementcited by 203
- 2024OmniParser for Pure Vision Based GUI Agentcited by 193
- 2023You Only Look at Screens: Multimodal Chain-of-Action Agentscited by 185
- 2023Synapse: Trajectory-as-Exemplar Prompting with Memory for Computer Controlcited by 180
- 2024Ferret-UI: Grounded Mobile UI Understanding with Multimodal LLMscited by 173
- 2025UI-R1: Enhancing Efficient Action Prediction of GUI Agents by Reinforcement Learningcited by 166
- 2024OS-Genesis: Automating GUI Agent Trajectory Construction via Reverse Task Synthesiscited by 155
- 2025OpenCUA: Open Foundations for Computer-Use Agentscited by 151
- 2024UFO: A UI-Focused Agent for Windows OS Interactioncited by 147
- 2024Cradle: Empowering Foundation Agents Towards General Computer Controlcited by 114
- 2024GUI Agents with Foundation Models: A Comprehensive Surveycited by 110
- 2024ScreenAgent: A Vision Language Model-driven Computer Control Agentcited by 101
- 2024GUI Agents: A Surveycited by 101
- 2024ScreenAI: A Vision-Language Model for UI and Infographics Understandingcited by 97
- 2021AndroidEnv: A Reinforcement Learning Platform for Androidcited by 90
- 2025MAI-UI Technical Report: Real-World Centric Foundation GUI Agentscited by 88
- 2025A Survey of WebAgents: Towards Next-Generation AI Agents for Web Automation with Large Foundation Modelscited by 87
- 2024WorkArena++: Towards Compositional Planning and Reasoning-based Common Knowledge Work Taskscited by 81
- 2025Enhancing Visual Grounding for GUI Agents via Self-Evolutionary Reinforcement Learningcited by 79
- 2026Mobile-Agent-v3.5: Multi-platform Fundamental GUI Agentscited by 72
- 2025InfiGUIAgent: A Multimodal Generalist GUI Agent with Native Reasoning and Reflectioncited by 67
- 2025A Survey on (M)LLM-Based GUI Agentscited by 61
- 2024MobileSafetyBench: Evaluating Safety of Autonomous Agents in Mobile Device Controlcited by 49
- 2025AppAgentX: Evolving GUI Agents as Proficient Smartphone Userscited by 44
- 2025Web-Shepherd: Advancing PRMs for Reinforcing Web Agentscited by 41
- 2025TongUI: Internet-Scale Trajectories from Multimodal Web Tutorials for Generalized GUI Agentscited by 40
- 2025Step-GUI Technical Reportcited by 38
- 2025BadVLA: Towards Backdoor Attacks on Vision-Language-Action Models via Objective-Decoupled Optimizationcited by 37
- 2025Towards Trustworthy GUI Agents: A Surveycited by 33
- 2025ViMo: A Generative Visual GUI World Model for App Agentscited by 33
- 2025UI-Genie: A Self-Improving Approach for Iteratively Boosting MLLM-based Mobile GUI Agentscited by 33
50 most cited of 119 All 119 (JSON)
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2026ADeptS-Bench: Measuring the Trustworthiness of Computer Use Agents Across Devices
- 2025UI-CUBE: Enterprise-Grade Computer Use Agent Benchmarking Beyond Task Accuracy to Operational Reliability
- 2026Computer Use at the Edge of the Statistical Precipice
- 2026On the Reliability of Computer Use Agents
- 2026When Agents Fail to Act: A Diagnostic Framework for Tool Invocation Reliability in Multi-Agent LLM Systems
- 2026How Benchmarks Mis-Score Computer-Use Agents
- 2026CUA-SWE: When Computer-Use Agents Meet Visual Software Engineering
- 2026ERPBench: A State-Grounded Evaluation Paradigm for Computer-Use Agents in Enterprise Software