Owner-Harm: A Missing Threat Model for AI Agent Safety
On the map: LLM agents and tool use → Agent threats, guardrails and governance
Abstract (arXiv)
Existing AI agent safety benchmarks focus on generic criminal harm (cybercrime, harassment, weapon synthesis), leaving a systematic blind spot for a distinct and commercially consequential threat category: agents harming their own deployers. Real-world incidents illustrate the gap: Slack AI credential exfiltration (Aug 2024), Microsoft 365 Copilot calendar-injection leaks (Jan 2024), and a Meta agent unauthorized forum post exposing operational data (Mar 2026). We propose Owner-Harm, a formal threat model with eight categories of agent behavior damaging the deployer. We quantify the defense gap on two benchmarks: a compositional safety system achieves 100% TPR / 0% FPR on AgentHarm (generic criminal harm) yet only 14.8% (4/27; 95% CI: 5.9%-32.5%) on AgentDojo injection tasks (prompt-injection-mediated owner harm). A controlled generic-LLM baseline shows the gap is not inherent to owner-harm (62.7% vs. 59.3%, delta 3.4 pp) but arises from environment-bound symbolic rules that fail to generalize across tool vocabularies. On a post-hoc 300-scenario owner-harm benchmark, the gate alone achieves 75.3% TPR / 3.3% FPR; adding a deterministic post-audit verifier raises overall TPR to 85.3% (+10.0 pp) and Hijacking detection from 43.3% to 93.3%, demonstrating strong layer complementarity. We introduce the Symbolic-Semantic Defense Generalization (SSDG) framework relating information coverage to detection rate. Two SSDG experiments partially validate it: context deprivation amplifies the detection gap 3.4x (R = 3.60 vs. R = 1.06); context injection reveals structured goal-action alignment, not text concatenation, is required for effective owner-harm detection.
Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Placed in Agent threats, guardrails and governance, one of 16 directions in LLM agents and tool use. Builds on 14 of its 31 references that are in the map. First posted on arXiv 20 Apr 2026.
Builds on
14 of 31 references.
Closest ancestors · 8
Same direction, newest first
- 2026Solver-Aided Verification of Policy Compliance in Tool-Augmented LLM Agents20 Mar
- 2026Formal Policy Enforcement for Real-World Agentic Systems18 Feb
- 2026Towards Verifiably Safe Tool Use for LLM Agents12 Jan
- 2025ShieldAgent: Shielding Agents via Verifiable Safety Policy Reasoning26 Mar
- 2025AgentSpec: Customizable Runtime Enforcement for Safe and Reliable LLM Agents24 Mar
- 2025AGrail: A Lifelong Agent Guardrail with Effective and Adaptive Safety Detection17 Feb
- 2024AgentHarm: A Benchmark for Measuring Harmfulness of LLM Agents11 Oct
- 2023Identifying the Risks of LM Agents with an LM-Emulated Sandbox25 Sep
Other references · 6
- 2023Not what you've signed up for: Compromising Real-World LLM-Integrated Applications with Indirect Prompt Injectioncited by 1,339
- 2023Llama Guard: LLM-based Input-Output Safeguard for Human-AI Conversationscited by 1,124
- 2023NeMo Guardrails: A Toolkit for Controllable and Safe LLM Applications with Programmable Railscited by 347
- 2023Benchmarking and Defending Against Indirect Prompt Injection Attacks on Large Language Modelscited by 237
- 2024ShieldGemma: Generative AI Content Moderation Based on Gemmacited by 224
- 2023Provably safe systems: the only path to controllable AGIcited by 29
Built on by
Citing papers per year: 2026: 1.
1 paper.
Similar, not cited
Close in wording, no citation either way
- 2026The Blind Spot of Agent Safety: How Benign User Instructions Expose Critical Vulnerabilities in Computer-Use Agents
- 2026AgentHazard: A Benchmark for Evaluating Harmful Behavior in Computer-Use Agents
- 2026AgentTrust: Runtime Safety Evaluation and Interception for AI Agent Tool Use
- 2026Agent Safety Should Be a Runtime Contract
- 2025A Safety and Security Framework for Real-World Agentic Systems
- 2026Taxonomy and Consistency Analysis of Safety Benchmarks for AI Agents
- 2025OpenAgentSafety: A Comprehensive Framework for Evaluating Real-World AI Agent Safety
- 2024Safeguarding AI Agents: Developing and Analyzing Safety Architectures