# A Systematic Security Evaluation of OpenClaw and Its Variants

arXiv: https://arxiv.org/abs/2604.03131 · HTML: https://mlsanity.com/paper/2604.03131/

## Key facts

- Cited by 13 papers in the map (data through 14 Oct 2026).
- Builds on 13 of its 13 references that are in the map.
- First posted on arXiv 3 Apr 2026.

## Builds on (most cited first, 13 of 13)

- [AgentHarm: A Benchmark for Measuring Harmfulness of LLM Agents](https://mlsanity.com/paper/2410.09024/) (2024)
- [AgentPoison: Red-teaming LLM Agents via Poisoning Memory or Knowledge Bases](https://mlsanity.com/paper/2407.12784/) (2024)
- [Agent Security Bench (ASB): Formalizing and Benchmarking Attacks and Defenses in LLM-based Agents](https://mlsanity.com/paper/2410.02644/) (2024)
- [OS-Harm: A Benchmark for Measuring Safety of Computer Use Agents](https://mlsanity.com/paper/2506.14866/) (2025)
- [MCPTox: A Benchmark for Tool Poisoning Attack on Real-World MCP Servers](https://mlsanity.com/paper/2508.14925/) (2025)
- [Memory Poisoning Attack and Defense on Memory Based LLM-Agents](https://mlsanity.com/paper/2601.05504/) (2026)
- [Defending Against Prompt Injection with DataFilter](https://mlsanity.com/paper/2510.19207/) (2025)
- [From Assistant to Double Agent: Formalizing and Benchmarking Attacks on OpenClaw for Personalized Local AI Agent](https://mlsanity.com/paper/2602.08412/) (2026)
- [Don't Let the Claw Grip Your Hand: A Security Analysis and Defense Framework for OpenClaw](https://mlsanity.com/paper/2603.10387/) (2026)
- [Taming OpenClaw: Security Analysis and Mitigation of Autonomous LLM Agent Threats](https://mlsanity.com/paper/2603.11619/) (2026)
- [Uncovering Security Threats and Architecting Defenses in Autonomous Agents: A Case Study of OpenClaw](https://mlsanity.com/paper/2603.12644/) (2026)
- [ClawKeeper: Comprehensive Safety Protection for OpenClaw Agents Through Skills, Plugins, and Watchers](https://mlsanity.com/paper/2603.24414/) (2026)
- [MCP-ITP: An Automated Framework for Implicit Tool Poisoning in MCP](https://mlsanity.com/paper/2601.07395/) (2026)

## Built on by (most cited first, 13 of 13)

- [Benign in Isolation, Harmful in Composition: Security Risks in Agent Skill Ecosystems](https://mlsanity.com/paper/2606.15242/) (2026)
- [Auditing Agent Harness Safety](https://mlsanity.com/paper/2605.14271/) (2026)
- [From Chatbot to Digital Colleague: The Paradigm Shift Toward Persistent Autonomous AI](https://mlsanity.com/paper/2606.14502/) (2026)
- [Understanding and Evaluating Claw-like Agent Security Through a Computer-Systems Lens](https://mlsanity.com/paper/2606.30755/) (2026)
- [BraveGuard: From Open-World Threats to Safer Computer-Use Agents](https://mlsanity.com/paper/2606.01166/) (2026)
- [Red-Teaming Agent Execution Contexts: Open-World Security Evaluation on OpenClaw](https://mlsanity.com/paper/2605.11047/) (2026)
- [ASEval: Automated Trajectory-Level Security Testing for Autonomous Agents](https://mlsanity.com/paper/2605.22321/) (2026)
- [Constraining Host-Level Abuse in Self-Hosted Computer-Use Agents via TEE-Backed Isolation](https://mlsanity.com/paper/2605.06393/) (2026)
- [Dynamic Capability Scoping for Enterprise AI Agents: A Synthetic Dataset and Three-Source Permission Architecture](https://mlsanity.com/paper/2607.22445/) (2026)
- [Understanding and mitigating the risks of OpenClaw for non-technical users: A practical guide with Skill](https://mlsanity.com/paper/2606.11007/) (2026)
- [Empirical Evaluation of Task-Based Permission Scoping Architecture for AI Agents](https://mlsanity.com/paper/2609.15422/) (2026)
- [Deny Without Disabling: Authorization-Paired Evaluation and Control for Multi-Agent Systems](https://mlsanity.com/paper/2610.00371/) (2026)
- [Complex Agents, Shallow Tests: Demystifying and Enhancing Test Adequacy of Agent Harness in the Wild](https://mlsanity.com/paper/2610.04921/) (2026)

## Abstract (arXiv)

Tool-augmented AI agents substantially extend the practical capabilities of large language models, but they also introduce security risks that cannot be identified through model-only evaluation. In this paper, we present a systematic security assessment of six representative OpenClaw-series agent frameworks, namely OpenClaw, AutoClaw, QClaw, KimiClaw, MaxClaw, and ArkClaw, under multiple backbone models. To support this study, we construct a benchmark of 205 test cases covering representative attack behaviors across the full agent execution lifecycle, enabling unified evaluation of risk exposure at both the framework and model levels. Our results show that all evaluated agents exhibit substantial security vulnerabilities, and that agentized systems are significantly riskier than their underlying models used in isolation. In particular, reconnaissance and discovery behaviors emerge as the most common weaknesses, while different frameworks expose distinct high-risk profiles, including credential leakage, lateral movement, privilege escalation, and resource development. These findings indicate that the security of modern agent systems is shaped not only by the safety properties of the backbone model, but also by the coupling among model capability, tool use, multi-step planning, and runtime orchestration. We further show that once an agent is granted execution capability and persistent runtime context, weaknesses arising in early stages can be amplified into concrete system-level failures. Overall, our study highlights the need to move beyond prompt-level safeguards toward lifecycle-wide security governance for intelligent agent frameworks.