mlsanityData to 14 Oct 2026

OS-Harm: A Benchmark for Measuring Safety of Computer Use Agents

arXiv 2506.14866 ↗First posted 17 Jun 2025cs.SE cs.LGcited by 111

On the map: LLM agents and tool use → Agent threats, guardrails and governance

also in 1 more
Abstract (arXiv)

Computer use agents are LLM-based agents that can directly interact with a graphical user interface, by processing screenshots or accessibility trees. While these systems are gaining popularity, their safety has been largely overlooked, despite the fact that evaluating and understanding their potential for harmful behavior is essential for widespread adoption. To address this gap, we introduce OS-Harm, a new benchmark for measuring safety of computer use agents. OS-Harm is built on top of the OSWorld environment and aims to test models across three categories of harm: deliberate user misuse, prompt injection attacks, and model misbehavior. To cover these cases, we create 150 tasks that span several types of safety violations (harassment, copyright infringement, disinformation, data exfiltration, etc.) and require the agent to interact with a variety of OS applications (email client, code editor, browser, etc.). Moreover, we propose an automated judge to evaluate both accuracy and safety of agents that achieves high agreement with human annotations (0.76 and 0.79 F1 score). We evaluate computer use agents based on a range of frontier models - such as o4-mini, Claude 3.7 Sonnet, Gemini 2.5 Pro - and provide insights into their safety. In particular, all models tend to directly comply with many deliberate misuse queries, are relatively vulnerable to static prompt injections, and occasionally perform unsafe actions. The OS-Harm benchmark is available at https://github.com/tml-epfl/os-harm.

Key facts. Cited by 111 papers in the map (data through 14 Oct 2026). Placed in Agent threats, guardrails and governance, one of 16 directions in LLM agents and tool use. Builds on 32 of its 46 references that are in the map. First posted on arXiv 17 Jun 2025.

Builds on

32 of 46 references.

Closest ancestors · 15

Same direction, newest first

  1. 2025WASP: Benchmarking Web Agent Security Against Prompt Injection Attacks22 Apr
  2. 2025Defeating Prompt Injections by Design24 Mar
  3. 2025AgentDAM: Privacy Leakage Evaluation for Autonomous Web Agents12 Mar
  4. 2025In-Context Defense in Computer Agents: An Empirical Study12 Mar
  5. 2025SafeArena: Evaluating the Safety of Autonomous Web Agents6 Mar
  6. 2025Commercial LLM Agents Are Already Vulnerable to Simple Yet Dangerous Attacks12 Feb
  7. 2024Attacking Vision-Language Computer Agents via Pop-ups4 Nov
  8. 2024Refusal-Trained LLMs Are Easily Jailbroken As Browser Agents11 Oct
  9. 2024AgentHarm: A Benchmark for Measuring Harmfulness of LLM Agents11 Oct
  10. 2024Agent Security Bench (ASB): Formalizing and Benchmarking Attacks and Defenses in LLM-based Agents3 Oct
  11. 2024AgentDojo: A Dynamic Environment to Evaluate Prompt Injection Attacks and Defenses for LLM Agents19 Jun
  12. 2024Dissecting Adversarial Robustness of Multimodal LM Agents18 Jun
3 more
  1. 2024TrustAgent: Towards Safe and Trustworthy LLM-based Agents2 Feb
  2. 2024R-Judge: Benchmarking Safety Risk Awareness for LLM Agents18 Jan
  3. 2023Identifying the Risks of LM Agents with an LM-Emulated Sandbox25 Sep

Other references · 17

  1. 2023Toolformer: Language Models Can Teach Themselves to Use Toolscited by 4,182
  2. 2023Universal and Transferable Adversarial Attacks on Aligned Language Modelscited by 3,104
  3. 2022Beyond the Imitation Game: Quantifying and extrapolating the capabilities of language modelscited by 1,998
  4. 2021WebGPT: Browser-assisted question-answering with human feedbackcited by 1,745
  5. 2024SWE-agent: Agent-Computer Interfaces Enable Automated Software Engineeringcited by 1,737
  6. 2023Not what you've signed up for: Compromising Real-World LLM-Integrated Applications with Indirect Prompt Injectioncited by 1,443
  7. 2024HarmBench: A Standardized Evaluation Framework for Automated Red Teaming and Robust Refusalcited by 1,429
  8. 2024OSWorld: Benchmarking Multimodal Agents for Open-Ended Tasks in Real Computer Environmentscited by 1,195
  9. 2024GPT-4V(ision) is a Generalist Web Agent, if Groundedcited by 563
  10. 2024JailbreakBench: An Open Robustness Benchmark for Jailbreaking Large Language Modelscited by 536
  11. 2024Jailbreaking Leading Safety-Aligned LLMs with Simple Adaptive Attackscited by 486
  12. 2024Agent-as-a-Judge: Evaluate Agents with Agentscited by 238
5 more
  1. 2023Can LLMs Keep a Secret? Testing Privacy Implications of Language Models via Contextual Integrity Theorycited by 230
  2. 2024PrivacyLens: Evaluating Privacy Norm Awareness of Language Models in Actioncited by 171
  3. 2024AI Sandbagging: Language Models can Strategically Underperform on Evaluationscited by 145
  4. 2024Autonomous Evaluation and Refinement of Digital Agentscited by 137
  5. 2025AgentRewardBench: Evaluating Automatic Evaluations of Web Agent Trajectoriescited by 108

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 13; 2026: 98.

111 papers.

Sort:

By area:

  1. 2026Skill-Inject: Measuring Agent Vulnerability to Skill File Attackscited by 90
  2. 2026OSWorld 2.0: Benchmarking Computer Use Agents on Long-Horizon Real-World Taskscited by 40
  3. 2026ATBench: A Diverse and Realistic Agent Trajectory Benchmark for Safety Evaluation and Diagnosiscited by 36
  4. 2026ClawsBench: Evaluating Capability and Safety of LLM Productivity Agents in Simulated Workspacescited by 33
  5. 2025Reliable Weak-to-Strong Monitoring of LLM Agentscited by 31
  6. 2025OSWorld-MCP: Benchmarking MCP Tool Invocation In Computer-Use Agentscited by 29
  7. 2025Agentic AI Security: Threats, Defenses, Evaluation, and Open Challengescited by 28
  8. 2026MUZZLE: Adaptive Agentic Red-Teaming of Web Agents Against Indirect Prompt Injection Attackscited by 21
  9. 2025OS-Sentinel: Towards Safety-Enhanced Mobile GUI Agents via Hybrid Validation in Realistic Workflowscited by 19
  10. 2026CaMeLs Can Use Computers Too: System-level Security for Computer Use Agentscited by 17
  11. 2026How Vulnerable Are AI Agents to Indirect Prompt Injections? Insights from a Large-Scale Public Competitioncited by 17
  12. 2025Are Your Agents Upward Deceivers?cited by 16
38 more
  1. 2025A Benchmark for Evaluating Outcome-Driven Constraint Violations in Autonomous AI Agentscited by 13
  2. 2026A Systematic Security Evaluation of OpenClaw and Its Variantscited by 13
  3. 2025GhostEI-Bench: Do Mobile Agents Resilience to Environmental Injection in Dynamic On-Device Environments?cited by 12
  4. 2026SafePred: A Predictive Guardrail for Computer-Using Agents via World Modelscited by 12
  5. 2026The Blind Spot of Agent Safety: How Benign User Instructions Expose Critical Vulnerabilities in Computer-Use Agentscited by 12
  6. 2026From Question Answering to Task Completion: A Survey on Agent System and Harness Designcited by 12
  7. 2026Auditing Agent Harness Safetycited by 11
  8. 2025It's a TRAP! Task-Redirecting Agent Persuasion Benchmark for Web Agentscited by 10
  9. 2026SafePro: Evaluating the Safety of Professional-Level AI Agentscited by 10
  10. 2025Just Do It!? Computer-Use Agents Exhibit Blind Goal-Directednesscited by 9
  11. 2026LPS-Bench: Benchmarking Safety Awareness of Computer-Use Agents in Long-Horizon Planning under Benign and Adversarial Scenarioscited by 9
  12. 2026When Actions Go Off-Task: Detecting and Correcting Misaligned Actions in Computer-Use Agentscited by 8
  13. 2026Adaptive Vision-Language Model Routing for Computer Use Agentscited by 8
  14. 2026AgentDoG 1.5: A Lightweight and Scalable Alignment Framework for AI Agent Safety and Securitycited by 8
  15. 2026Qwen-CUA: Native Computer Use for (almost) Everythingcited by 8
  16. 2026Visual Confused Deputy: Exploiting and Defending Perception Failures in Computer-Using Agentscited by 7
  17. 2026CORA: Conformal Risk-Controlled Agents for Safeguarded Mobile GUI Automationcited by 7
  18. 2026Toward Secure LLM Agents: Threat Surfaces, Attacks, Defenses, and Evaluationcited by 7
  19. 2026Your Agentic LLMs Secretly Encode Indirect Prompt-Injection Exposure in Hidden Statescited by 7
  20. 2026EntWorld: A Holistic Environment and Benchmark for Verifiable Enterprise GUI Agentscited by 6
  21. 2026Do Coding Agents Understand Least-Privilege Authorization?cited by 6
  22. 2026SkillHarm: Lifecycle-Aware Skill-Based Attacks via Automated Constructioncited by 6
  23. 2026From Risk Classification to Action Plan Remediation: A Guardrail Feedback Driven Framework for LLM Agentscited by 6
  24. 2025Your Harness is Not Secure: Benchmarking Real-world Threat of Command Line Interface Agentcited by 5
  25. 2026MirrorGuard: Toward Secure Computer-Use Agents via Simulation-to-Real Reasoning Correctioncited by 5
  26. 2026LITMUS: Benchmarking Behavioral Jailbreaks of LLM Agents in Real OS Environmentscited by 5
  27. 2025SWIRL: A Staged Workflow for Interleaved Reinforcement Learning in Mobile GUI Controlcited by 4
  28. 2026From Biased Chatbots to Biased Agents: Examining Role Assignment Effects on LLM Agent Robustnesscited by 3
  29. 2026MCPShield: A Security Cognition Layer for Adaptive Trust Calibration in Model Context Protocol Agentscited by 3
  30. 2026Preference Redirection via Attention Concentration: An Attack on Computer Use Agentscited by 3
  31. 2026Architecture Matters for Multi-Agent Securitycited by 3
  32. 2026GUI Agents with Reinforcement Learning: Toward Digital Inhabitantscited by 3
  33. 2026MOSAIC-Bench: Measuring Compositional Vulnerability Induction in Coding Agentscited by 3
  34. 2026Responsible Agentic AI Requires Explicit Provenancecited by 3
  35. 2026LivePI: More Realistic Benchmarking of Agents Against Indirect Prompt Injectioncited by 3
  36. 2026Decomposing and Measuring Evaluation Awarenesscited by 3
  37. 2026Untrusted Content Masking for Web Agents with Security Guaranteescited by 3
  38. 2026AgentS4D: Benchmarking Runtime Risks across the Execution Lifecycle of LLM-Based Workspace Agentscited by 3

50 most cited of 111 All 111 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2026When Benign Inputs Lead to Severe Harms: Eliciting Unsafe Unintended Behaviors of Computer-Use Agents
  2. 2026AgentHazard: A Benchmark for Evaluating Harmful Behavior in Computer-Use Agents
  3. 2025Measuring Harmfulness of Computer-Using Agents
  4. 2026OSWorld-Pro: Process-based Evaluation for Computer Use Agents
  5. 2026Owner-Harm: A Missing Threat Model for AI Agent Safety
  6. 2026Capable but Careless: Do Computer-Use Agents Follow Contextual Integrity?
  7. 2026Towards a Unified Misuse Monitoring Benchmark
  8. 2025SusBench: An Online Benchmark for Evaluating Dark Pattern Susceptibility of Computer-Use Agents