mlsanityData to 7 Oct 2026

Just Do It!? Computer-Use Agents Exhibit Blind Goal-Directedness

arXiv 2510.01670 ↗First posted 2 Oct 2025cs.AI cs.CL cs.CRcited by 8

On the map: LLM agents and tool use → Agent threats, guardrails and governance

Abstract (arXiv)

Computer-Use Agents (CUAs) are an increasingly deployed class of agents that take actions on GUIs to accomplish user goals. In this paper, we show that CUAs consistently exhibit Blind Goal-Directedness (BGD): a bias to pursue goals regardless of feasibility, safety, reliability, or context. We characterize three prevalent patterns of BGD: (i) lack of contextual reasoning, (ii) assumptions and decisions under ambiguity, and (iii) contradictory or infeasible goals. We develop BLIND-ACT, a benchmark of 90 tasks capturing these three patterns. Built on OSWorld, BLIND-ACT provides realistic environments and employs LLM-based judges to evaluate agent behavior, achieving 93.75% agreement with human annotations. We use BLIND-ACT to evaluate nine frontier models, including Claude Sonnet and Opus 4, Computer-Use-Preview, and GPT-5, observing high average BGD rates (80.8%) across them. We show that BGD exposes subtle risks that arise even when inputs are not directly harmful. While prompting-based interventions lower BGD levels, substantial risk persists, highlighting the need for stronger training- or inference-time interventions. Qualitative analysis reveals observed failure modes: execution-first bias (focusing on how to act over whether to act), thought-action disconnect (execution diverging from reasoning), and request-primacy (justifying actions due to user request). Identifying BGD and introducing BLIND-ACT establishes a foundation for future research on studying and mitigating this fundamental risk and ensuring safe CUA deployment.

Key facts. Cited by 8 papers in the map (data through 7 Oct 2026). Placed in Agent threats, guardrails and governance, one of 16 directions in LLM agents and tool use. Builds on 32 of its 41 references that are in the map. First posted on arXiv 2 Oct 2025.

Builds on

32 of 41 references.

Closest ancestors · 13

Same direction, newest first

  1. 2025A Systematization of Security Vulnerabilities in Computer Use Agents7 Jul
  2. 2025OS-Harm: A Benchmark for Measuring Safety of Computer Use Agents17 Jun
  3. 2025VPI-Bench: Visual Prompt Injection Attacks for Computer-Use Agents3 Jun
  4. 2025MLA-Trust: Benchmarking Trustworthiness of Multimodal LLM Agents in GUI Environments2 Jun
  5. 2025RiOSWorld: Benchmarking the Risk of Multimodal Computer-Use Agents31 May
  6. 2025RedTeamCUA: Realistic Adversarial Testing of Computer-Use Agents in Hybrid Web-OS Environments28 May
  7. 2025A Survey on the Safety and Security Threats of Computer-Using Agents: JARVIS or Ultron?16 May
  8. 2025WASP: Benchmarking Web Agent Security Against Prompt Injection Attacks22 Apr
  9. 2025DoomArena: A framework for Testing AI Agents Against Evolving Security Threats18 Apr
  10. 2025SafeArena: Evaluating the Safety of Autonomous Web Agents6 Mar
  11. 2025Why Are Web AI Agents More Vulnerable Than Standalone LLMs? A Security Analysis27 Feb
  12. 2024Dissecting Adversarial Robustness of Multimodal LM Agents18 Jun
1 more
  1. 2023Identifying the Risks of LM Agents with an LM-Emulated Sandbox25 Sep

Other references · 19

  1. 2024The Llama 3 Herd of Modelscited by 14,192
  2. 2025DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learningcited by 6,146
  3. 2025Qwen2.5-VL Technical Reportcited by 5,418
  4. 2023Jailbroken: How Does LLM Safety Training Fail?cited by 1,565
  5. 2024OSWorld: Benchmarking Multimodal Agents for Open-Ended Tasks in Real Computer Environmentscited by 1,134
  6. 2025UI-TARS: Pioneering Automated GUI Interaction with Native Agentscited by 584
  7. 2024AndroidWorld: A Dynamic Benchmarking Environment for Autonomous Agentscited by 450
  8. 2024WebVoyager: Building an End-to-End Web Agent with Large Multimodal Modelscited by 424
  9. 2023Set-of-Mark Prompting Unleashes Extraordinary Visual Grounding in GPT-4Vcited by 291
  10. 2023Jailbreak in pieces: Compositional Adversarial Attacks on Multi-Modal Language Modelscited by 260
  11. 2024Windows Agent Arena: Evaluating Multi-Modal OS Agents at Scalecited by 212
  12. 2025UI-TARS-2 Technical Report: Advancing GUI Agent with Multi-Turn Reinforcement Learningcited by 205
7 more
  1. 2025An Illusion of Progress? Assessing the Current State of Web Agentscited by 165
  2. 2024PrivacyLens: Evaluating Privacy Norm Awareness of Language Models in Actioncited by 161
  3. 2024Large Language Model-Brained GUI Agents: A Surveycited by 160
  4. 2024The BrowserGym Ecosystem for Web Agent Researchcited by 152
  5. 2025OpenCUA: Open Foundations for Computer-Use Agentscited by 151
  6. 2025Towards Trustworthy GUI Agents: A Surveycited by 33
  7. 2025sudo rm -rf agentic_securitycited by 8

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 1; 2026: 7.

8 papers.

Similar, not cited

Close in wording, no citation either way

  1. 2026CUAAudit: Meta-Evaluation of Vision-Language Models as Auditors of Autonomous Computer-Use Agents
  2. 2026CAVEAT: Towards Robust Computer-Use Agents in Incentive-Misaligned Environments
  3. 2026Capable but Careless: Do Computer-Use Agents Follow Contextual Integrity?
  4. 2026ADeptS-Bench: Measuring the Trustworthiness of Computer Use Agents Across Devices
  5. 2026Step-level Optimization for Efficient Computer-use Agents
  6. 2026When Actions Go Off-Task: Detecting and Correcting Misaligned Actions in Computer-Use Agents
  7. 2026When Benign Inputs Lead to Severe Harms: Eliciting Unsafe Unintended Behaviors of Computer-Use Agents
  8. 2026A Dual-Process Perspective on Nudge Susceptibility in LLM-Based GUI Agents