mlsanityData to 7 Oct 2026

Securing Computer-Use Agents: A Unified Architecture-Lifecycle Framework for Deployment-Grounded Reliability

arXiv 2605.07110 ↗First posted 8 May 2026cs.CL cs.SEcited by 0

On the map: LLM agents and tool use → Agent threats, guardrails and governance

Abstract (arXiv)

Computer-use agents(CUAs)are moving frombounded benchmarks toward real software environments, wherethey operate browsers, desktops, mobile applications, flesystems,terminals, and tool backends. In such settings, reliability isno longer captured by task success alone: perception errors,planning drift, memory use, tool mediation, permission scope,and runtime oversight jointly determine whether agent actionsremain aligned with user intent, Existing surveys organize theCUA landscape by methods, platforms, benchmarks, or securitythreats, but less explicitly connect capability formation, author-ity exposure, failure manifestation, and control placement. Toaddress this gap, the article develops an architecture-lifecycleframework for deployment-grounded reliability in CUAs. Thearchitectural view analyzes Perception, Decision, and Executionas coupled layers that transform software observations intoauthority-bearing actions, The lifecycle view examines Creation.Deployment, Operation, and Maintenance as stages in which priorsare learned, tools and permissions are bound, runtime trajecto.ries are stressed, and assurance must be preserved under drift.Using this lens, the analysis synthesizes representative systems,benchmarks, and security/privacy studies; distinguishes wherefailures become visible from where their enabling conditions areintroduced, and maps recurring intervention surfaces for controloversight, and assurance. OpenClaw is used only as a public moti.vating example of an open deployment pattern, not as a verifedinternal case study. The conclusion highlights open challengesin controllable grounding, long-horizon constraint preservation,safe authority binding, mixed-trust runtime defense, privacy-preserving memory,and continual assurance.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Agent threats, guardrails and governance, one of 16 directions in LLM agents and tool use. Builds on 175 of its 191 references that are in the map. First posted on arXiv 8 May 2026.

Builds on

175 of 191 references.

Closest ancestors · 56

Same direction, newest first

  1. 2026The Blind Spot of Agent Safety: How Benign User Instructions Expose Critical Vulnerabilities in Computer-Use Agents12 Apr
  2. 2026Preference Redirection via Attention Concentration: An Attack on Computer Use Agents9 Apr
  3. 2026Are GUI Agents Focused Enough? Automated Distraction via Semantic-level UI Element Injection9 Apr
  4. 2026WebSP-Eval: Evaluating Web Agents on Website Security and Privacy Tasks7 Apr
  5. 2026Your Agent, Their Asset: A Real-World Safety Analysis of OpenClaw6 Apr
  6. 2026AgentRAE: Remote Action Execution through Notification-based Visual Backdoors against Screenshots-based Mobile GUI Agents24 Mar
  7. 2026SlowBA: An efficiency backdoor attack towards VLM-based GUI agents9 Mar
  8. 2026Dual-Modality Multi-Stage Adversarial Safety Training: Robustifying Multimodal Web Agents Against Cross-Modal Attacks4 Mar
  9. 2026When Actions Go Off-Task: Detecting and Correcting Misaligned Actions in Computer-Use Agents9 Feb
  10. 2026From Assistant to Double Agent: Formalizing and Benchmarking Attacks on OpenClaw for Personalized Local AI Agent9 Feb
  11. 2026When Benign Inputs Lead to Severe Harms: Eliciting Unsafe Unintended Behaviors of Computer-Use Agents9 Feb
  12. 2026WebSentinel: Detecting and Localizing Prompt Injection Attacks for Web Agents3 Feb
38 more
  1. 2026LPS-Bench: Benchmarking Safety Awareness of Computer-Use Agents in Long-Horizon Planning under Benign and Adversarial Scenarios3 Feb
  2. 2026Risky-Bench: Probing Agentic Safety Risks under Real-World Deployment3 Feb
  3. 2026AgentDoG: A Diagnostic Guardrail Framework for AI Agent Safety and Security26 Jan
  4. 2026MirrorGuard: Toward Secure Computer-Use Agents via Simulation-to-Real Reasoning Correction19 Jan
  5. 2026Mind the Gap: Action Rebinding Attacks against Android GUI Agents18 Jan
  6. 2026CaMeLs Can Use Computers Too: System-level Security for Computer Use Agents14 Jan
  7. 2026A Survey of Agentic AI and Cybersecurity: Challenges, Opportunities and Use-case Prototypes8 Jan
  8. 2025It's a TRAP! Task-Redirecting Agent Persuasion Benchmark for Web Agents29 Dec
  9. 2025DECEPTICON: How Dark Patterns Manipulate Web Agents28 Dec
  10. 2025Mitigating Indirect Prompt Injection via Instruction-Following Intent Analysis30 Nov
  11. 2025PropensityBench: Evaluating Latent Safety Risks in Large Language Models via an Agentic Approach24 Nov
  12. 2025GhostEI-Bench: Do Mobile Agents Resilience to Environmental Injection in Dynamic On-Device Environments?23 Oct
  13. 2025Investigating the Impact of Dark Patterns on LLM-Based Web Agents20 Oct
  14. 2025MCP Security Bench (MSB): Benchmarking Attacks Against Model Context Protocol in LLM Agents14 Oct
  15. 2025HackWorld: Evaluating Computer-Use Agents on Exploiting Web Application Vulnerabilities14 Oct
  16. 2025VeriGuard: Enhancing LLM Agent Safety via Verified Code Generation3 Oct
  17. 2025WAInjectBench: Benchmarking Prompt Injection Detections for Web Agents1 Oct
  18. 2025Secure and Efficient Access Control for Computer-Use Agents via Context Space26 Sep
  19. 2025Privacy in Action: Towards Realistic Privacy Mitigation and Evaluation for LLM-Powered Agents22 Sep
  20. 2025Environmental Injection Attacks against GUI Agents in Realistic Dynamic Environments14 Sep
  21. 2025AgentSentinel: An End-to-End and Real-Time Security Defense Framework for Computer-Use Agents9 Sep
  22. 2025Measuring Harmfulness of Computer-Using Agents31 Jul
  23. 2025A Systematization of Security Vulnerabilities in Computer Use Agents7 Jul
  24. 2025OS-Harm: A Benchmark for Measuring Safety of Computer Use Agents17 Jun
  25. 2025VPI-Bench: Visual Prompt Injection Attacks for Computer-Use Agents3 Jun
  26. 2025RiOSWorld: Benchmarking the Risk of Multimodal Computer-Use Agents31 May
  27. 2025RedTeamCUA: Realistic Adversarial Testing of Computer-Use Agents in Hybrid Web-OS Environments28 May
  28. 2025Hidden Ghost Hand: Unveiling Backdoor Vulnerabilities in MLLM-Powered Mobile GUI Agents20 May
  29. 2025WebInject: Prompt Injection Attack to Web Agents16 May
  30. 2025A Survey on the Safety and Security Threats of Computer-Using Agents: JARVIS or Ultron?16 May
  31. 2025WASP: Benchmarking Web Agent Security Against Prompt Injection Attacks22 Apr
  32. 2025Les Dissonances: Cross-Tool Harvesting and Polluting in Pool-of-Tools Empowered LLM Agents4 Apr
  33. 2025ShieldAgent: Shielding Agents via Verifiable Safety Policy Reasoning26 Mar
  34. 2025MIP against Agent: Malicious Image Patches Hijacking Multimodal OS Agents13 Mar
  35. 2025AgentDAM: Privacy Leakage Evaluation for Autonomous Web Agents12 Mar
  36. 2025In-Context Defense in Computer Agents: An Empirical Study12 Mar
  37. 2025Evaluating the Robustness of Multimodal Agents Against Active Environmental Injection Attacks18 Feb
  38. 2024Prompt Infection: LLM-to-LLM Prompt Injection within Multi-Agent Systems9 Oct

50 newest of 56 All 56 (JSON)

Other references · 119

  1. 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,355
  2. 2023Reflexion: Language Agents with Verbal Reinforcement Learningcited by 4,064
  3. 2023Tree of Thoughts: Deliberate Problem Solving with Large Language Modelscited by 3,155
  4. 2023Qwen-VL: A Versatile Vision-Language Model for Understanding, Localization, Text Reading, and Beyondcited by 2,179
  5. 2023WebArena: A Realistic Web Environment for Building Autonomous Agentscited by 1,908
  6. 2023Mind2Web: Towards a Generalist Agent for the Webcited by 1,210
  7. 2024OSWorld: Benchmarking Multimodal Agents for Open-Ended Tasks in Real Computer Environmentscited by 1,134
  8. 2023CogAgent: A Visual Language Model for GUI Agentscited by 692
  9. 2024SeeClick: Harnessing GUI Grounding for Advanced Visual GUI Agentscited by 553
  10. 2024WebVoyager: Building an End-to-End Web Agent with Large Multimodal Modelscited by 424
  11. 2023AppAgent: Multimodal Agents as Smartphone Userscited by 407
  12. 2023A Real-World WebAgent with Planning, Long Context Understanding, and Program Synthesiscited by 350
38 more
  1. 2024WorkArena: How Capable Are Web Agents at Solving Common Knowledge Work Tasks?cited by 348
  2. 2023AgentTuning: Enabling Generalized Agent Abilities for LLMscited by 319
  3. 2023Android in the Wild: A Large-Scale Dataset for Android Device Controlcited by 314
  4. 2024Mobile-Agent: Autonomous Multi-Modal Mobile Device Agent with Visual Perceptioncited by 283
  5. 2025ScreenSpot-Pro: GUI Grounding for Professional High-Resolution Computer Usecited by 246
  6. 2025GUI-R1 : A Generalist R1-Style Vision-Language Action Model For GUI Agentscited by 240
  7. 2025UI-TARS-2 Technical Report: Advancing GUI Agent with Multi-Turn Reinforcement Learningcited by 205
  8. 2024OS-Copilot: Towards Generalist Computer Agents with Self-Improvementcited by 203
  9. 2024OmniParser for Pure Vision Based GUI Agentcited by 193
  10. 2023You Only Look at Screens: Multimodal Chain-of-Action Agentscited by 185
  11. 2023Synapse: Trajectory-as-Exemplar Prompting with Memory for Computer Controlcited by 180
  12. 2024Ferret-UI: Grounded Mobile UI Understanding with Multimodal LLMscited by 173
  13. 2025UI-R1: Enhancing Efficient Action Prediction of GUI Agents by Reinforcement Learningcited by 166
  14. 2024OS-Genesis: Automating GUI Agent Trajectory Construction via Reverse Task Synthesiscited by 155
  15. 2025OpenCUA: Open Foundations for Computer-Use Agentscited by 151
  16. 2024UFO: A UI-Focused Agent for Windows OS Interactioncited by 147
  17. 2024Cradle: Empowering Foundation Agents Towards General Computer Controlcited by 114
  18. 2024GUI Agents with Foundation Models: A Comprehensive Surveycited by 110
  19. 2024ScreenAgent: A Vision Language Model-driven Computer Control Agentcited by 101
  20. 2024GUI Agents: A Surveycited by 101
  21. 2024ScreenAI: A Vision-Language Model for UI and Infographics Understandingcited by 97
  22. 2021AndroidEnv: A Reinforcement Learning Platform for Androidcited by 90
  23. 2025MAI-UI Technical Report: Real-World Centric Foundation GUI Agentscited by 88
  24. 2025A Survey of WebAgents: Towards Next-Generation AI Agents for Web Automation with Large Foundation Modelscited by 87
  25. 2024WorkArena++: Towards Compositional Planning and Reasoning-based Common Knowledge Work Taskscited by 81
  26. 2025Enhancing Visual Grounding for GUI Agents via Self-Evolutionary Reinforcement Learningcited by 79
  27. 2026Mobile-Agent-v3.5: Multi-platform Fundamental GUI Agentscited by 72
  28. 2025InfiGUIAgent: A Multimodal Generalist GUI Agent with Native Reasoning and Reflectioncited by 67
  29. 2025A Survey on (M)LLM-Based GUI Agentscited by 61
  30. 2024MobileSafetyBench: Evaluating Safety of Autonomous Agents in Mobile Device Controlcited by 49
  31. 2025AppAgentX: Evolving GUI Agents as Proficient Smartphone Userscited by 44
  32. 2025Web-Shepherd: Advancing PRMs for Reinforcing Web Agentscited by 41
  33. 2025TongUI: Internet-Scale Trajectories from Multimodal Web Tutorials for Generalized GUI Agentscited by 40
  34. 2025Step-GUI Technical Reportcited by 38
  35. 2025BadVLA: Towards Backdoor Attacks on Vision-Language-Action Models via Objective-Decoupled Optimizationcited by 37
  36. 2025Towards Trustworthy GUI Agents: A Surveycited by 33
  37. 2025ViMo: A Generative Visual GUI World Model for App Agentscited by 33
  38. 2025UI-Genie: A Self-Improving Approach for Iteratively Boosting MLLM-based Mobile GUI Agentscited by 33

50 most cited of 119 All 119 (JSON)

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026ADeptS-Bench: Measuring the Trustworthiness of Computer Use Agents Across Devices
  2. 2025UI-CUBE: Enterprise-Grade Computer Use Agent Benchmarking Beyond Task Accuracy to Operational Reliability
  3. 2026Computer Use at the Edge of the Statistical Precipice
  4. 2026On the Reliability of Computer Use Agents
  5. 2026When Agents Fail to Act: A Diagnostic Framework for Tool Invocation Reliability in Multi-Agent LLM Systems
  6. 2026How Benchmarks Mis-Score Computer-Use Agents
  7. 2026CUA-SWE: When Computer-Use Agents Meet Visual Software Engineering
  8. 2026ERPBench: A State-Grounded Evaluation Paradigm for Computer-Use Agents in Enterprise Software