# From Assistant to Double Agent: Formalizing and Benchmarking Attacks on OpenClaw for Personalized Local AI Agent

arXiv: https://arxiv.org/abs/2602.08412 · HTML: https://mlsanity.com/paper/2602.08412/

## Key facts

- Cited by 29 papers in the map (data through 14 Oct 2026).
- Placed in Agent threats, guardrails and governance, one of 16 directions in LLM agents and tool use.
- Builds on 19 of its 22 references that are in the map.
- First posted on arXiv 9 Feb 2026.

## Placed in

- LLM agents and tool use / [Agent threats, guardrails and governance](https://mlsanity.com/agents/agent-threats-guardrails-and-governance/)

## Builds on (most cited first, 19 of 19)

- [Chain-of-Thought Prompting Elicits Reasoning in Large Language Models](https://mlsanity.com/paper/2201.11903/) (2022)
- [ReAct: Synergizing Reasoning and Acting in Language Models](https://mlsanity.com/paper/2210.03629/) (2022)
- [Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena](https://mlsanity.com/paper/2306.05685/) (2023)
- [Toolformer: Language Models Can Teach Themselves to Use Tools](https://mlsanity.com/paper/2302.04761/) (2023)
- [Generative Agents: Interactive Simulacra of Human Behavior](https://mlsanity.com/paper/2304.03442/) (2023)
- [A Survey on Large Language Model based Autonomous Agents](https://mlsanity.com/paper/2308.11432/) (2023)
- [MetaGPT: Meta Programming for A Multi-Agent Collaborative Framework](https://mlsanity.com/paper/2308.00352/) (2023)
- [Not what you've signed up for: Compromising Real-World LLM-Integrated Applications with Indirect Prompt Injection](https://mlsanity.com/paper/2302.12173/) (2023)
- [AgentBench: Evaluating LLMs as Agents](https://mlsanity.com/paper/2308.03688/) (2023)
- [Mind2Web: Towards a Generalist Agent for the Web](https://mlsanity.com/paper/2306.06070/) (2023)
- [GAIA: a benchmark for General AI Assistants](https://mlsanity.com/paper/2311.12983/) (2023)
- [BloombergGPT: A Large Language Model for Finance](https://mlsanity.com/paper/2303.17564/) (2023)
- [InjecAgent: Benchmarking Indirect Prompt Injections in Tool-Integrated Large Language Model Agents](https://mlsanity.com/paper/2403.02691/) (2024)
- [DriveGPT4: Interpretable End-to-end Autonomous Driving via Large Language Model](https://mlsanity.com/paper/2310.01412/) (2023)
- [AgentPoison: Red-teaming LLM Agents via Poisoning Memory or Knowledge Bases](https://mlsanity.com/paper/2407.12784/) (2024)
- [Agent Security Bench (ASB): Formalizing and Benchmarking Attacks and Defenses in LLM-based Agents](https://mlsanity.com/paper/2410.02644/) (2024)
- [Beyond Memorization: Violating Privacy Via Inference with Large Language Models](https://mlsanity.com/paper/2310.07298/) (2023)
- [BackdoorAgent: A Unified Framework for Backdoor Attacks on LLM-based Agents](https://mlsanity.com/paper/2601.04566/) (2026)
- [A Tool for In-depth Analysis of Code Execution Reasoning of Large Language Models](https://mlsanity.com/paper/2501.18482/) (2025)

## Built on by (most cited first, 29 of 29)

- [ClawsBench: Evaluating Capability and Safety of LLM Productivity Agents in Simulated Workspaces](https://mlsanity.com/paper/2604.05172/) (2026)
- [Taming OpenClaw: Security Analysis and Mitigation of Autonomous LLM Agent Threats](https://mlsanity.com/paper/2603.11619/) (2026)
- [ClawKeeper: Comprehensive Safety Protection for OpenClaw Agents Through Skills, Plugins, and Watchers](https://mlsanity.com/paper/2603.24414/) (2026)
- [ClawSafety: "Safe" LLMs, Unsafe Agents](https://mlsanity.com/paper/2604.01438/) (2026)
- [A Systematic Security Evaluation of OpenClaw and Its Variants](https://mlsanity.com/paper/2604.03131/) (2026)
- [LiveClawBench: Benchmarking LLM Agents on Complex, Real-World Assistant Tasks](https://mlsanity.com/paper/2604.13072/) (2026)
- [Trojan's Whisper: Stealthy Manipulation of OpenClaw through Injected Bootstrapped Guidance](https://mlsanity.com/paper/2603.19974/) (2026)
- [Toward Secure LLM Agents: Threat Surfaces, Attacks, Defenses, and Evaluation](https://mlsanity.com/paper/2606.10749/) (2026)
- [AgentWatcher: A Rule-based Prompt Injection Monitor](https://mlsanity.com/paper/2604.01194/) (2026)
- [Mind Your HEARTBEAT! Claw Background Execution Inherently Enables Silent Memory Pollution](https://mlsanity.com/paper/2603.23064/) (2026)
- [LivePI: More Realistic Benchmarking of Agents Against Indirect Prompt Injection](https://mlsanity.com/paper/2605.17986/) (2026)
- ["What Did It Actually Do?": Understanding Risk Awareness and Traceability for Computer-Use Agents](https://mlsanity.com/paper/2603.28551/) (2026)
- [ASEval: Automated Trajectory-Level Security Testing for Autonomous Agents](https://mlsanity.com/paper/2605.22321/) (2026)
- [SafeClawBench: Separating Semantic, Audit-Evidence, and Sandbox Harm in Tool-Using LLM Agents](https://mlsanity.com/paper/2606.18356/) (2026)
- [When Claws Remember but Do Not Tell: Stealthy Memory Injection in Persistent Personal Agents](https://mlsanity.com/paper/2607.05189/) (2026)
- [UniClawBench: A Universal Benchmark for Proactive Agents on Real-World Tasks](https://mlsanity.com/paper/2607.08768/) (2026)
- [HarnessRisk: A Lifecycle-Oriented Benchmark for Agent Harness Safety](https://mlsanity.com/paper/2608.17597/) (2026)
- [How do AI agents talk about science and research? An exploration of scientific discussions on Moltbook using BERTopic](https://mlsanity.com/paper/2603.11375/) (2026)
- [When Routine Chats Turn Toxic: Unintended Long-Term State Poisoning in Personalized Agents](https://mlsanity.com/paper/2605.06731/) (2026)
- [Security of OpenClaw Agents: Fundamentals, Attacks, and Countermeasures](https://mlsanity.com/paper/2605.25435/) (2026)
- [JANUS: Foreseeing Latent Risk for Long-Horizon Agent Safety](https://mlsanity.com/paper/2607.19913/) (2026)
- [Security Attack and Defense Strategies for Autonomous Agent Frameworks: A Layered Review with OpenClaw as a Case Study](https://mlsanity.com/paper/2604.27464/) (2026)
- [Securing Computer-Use Agents: A Unified Architecture-Lifecycle Framework for Deployment-Grounded Reliability](https://mlsanity.com/paper/2605.07110/) (2026)
- [ASTELD: A Six-Axis Classification Framework for Autonomous AI Agents - Design, Evaluation, and an OpenClaw Case Study](https://mlsanity.com/paper/2608.05201/) (2026)
- [SynChain: Inducing Computer-Use Agent Systems to Construct Their Own Attack Chains](https://mlsanity.com/paper/2608.06862/) (2026)
- [A Blind Trust, the Bloody Thrust: When Attacker-Controlled Hook Updates Steer AI Agent Harnesses towards Malicious Behaviors](https://mlsanity.com/paper/2609.03884/) (2026)
- [AgentKernel: The Trust-Native Agentic Operating System](https://mlsanity.com/paper/2609.29647/) (2026)
- [PACE: Provenance-Aware Capability Enforcement for Tool-Using LLM Agents](https://mlsanity.com/paper/2610.01349/) (2026)
- [Pincer: Resource Authorization for Agents using a Digital Twin](https://mlsanity.com/paper/2610.02569/) (2026)

## Abstract (arXiv)

Although large language model (LLM)-based agents, exemplified by OpenClaw, are increasingly evolving from task-oriented systems into personalized AI assistants for solving complex real-world tasks, their practical deployment also introduces severe security risks. However, existing agent security research and evaluation frameworks primarily focus on synthetic or task-centric settings, and thus fail to accurately capture the attack surface and risk propagation mechanisms of personalized agents in real-world deployments. To address this gap, we propose Personalized Agent Security Bench (PASB), an end-to-end security evaluation framework tailored for real-world personalized agents. Building upon existing agent attack paradigms, PASB incorporates personalized usage scenarios, realistic toolchains, and long-horizon interactions, enabling black-box, end-to-end security evaluation on real systems. Using OpenClaw as a representative case study, we systematically evaluate its security across multiple personalized scenarios, tool capabilities, and attack types. Our results indicate that OpenClaw exhibits critical vulnerabilities at different execution stages, including user prompt processing, tool usage, and memory retrieval, highlighting substantial security risks in personalized agent deployments. The code for the proposed PASB framework is available at https://github.com/AstorYH/PASB.