mlsanityData to 7 Oct 2026

Reframing LLM Agent Security as an Agent-Human Interaction Problem

arXiv 2605.24309 ↗First posted 23 May 2026cs.CRcited by 8

On the map: LLM agents and tool use → Agent threats, guardrails and governance

Abstract (arXiv)

We argue that LLM agent security is fundamentally an agent-human interaction (AHI) problem, not a purely algorithmic one. To substantiate this position, we conduct a systematic analysis of 59 academic papers, 21 production agent systems, and 26 security plugins as of April 2026. Our analysis reveals a striking pattern: the three widely deployed human-centric security mechanisms (policy specification, runtime approval, and scope configuration) dominate industry practice, each adopted by at least 14 of 21 systems (14, 15, and 16, respectively), while the categories most heavily studied in academia (intent anchoring and trust labeling) see zero production deployment. Yet current human participation mechanisms are far from satisfactory: they suffer from a fundamental trade-off between cognitive burden and security guarantees, leaving users caught between approval fatigue and uncontrolled agent autonomy. We make three contributions. First, through a systematic comparison of LLM-based and human-based intent alignment, we argue that human participation in agent security decisions is indispensable given current capabilities. Second, we quantify a pronounced industry-academia mismatch: the security mechanisms that practitioners actually deploy receive scant research attention, while the approaches that researchers favor remain undeployed. Third, we propose a three-direction research agenda and call for AHI security to be recognized as a first-class research citizen, one that demands its own design principles, evaluation methods, and theoretical foundations.

Key facts. Cited by 8 papers in the map (data through 7 Oct 2026). Placed in Agent threats, guardrails and governance, one of 16 directions in LLM agents and tool use. Builds on 38 of its 64 references that are in the map. First posted on arXiv 23 May 2026.

Builds on

38 of 64 references.

Closest ancestors · 19

Same direction, newest first

  1. 2026Architecting Secure AI Agents: Perspectives on System-Level Defenses Against Indirect Prompt Injection Attacks31 Mar
  2. 2026Agent-Sentry: Bounding LLM Agents via Execution Provenance24 Mar
  3. 2026The Landscape of Prompt Injection Threats in LLM Agents: From Taxonomy to Analysis11 Feb
  4. 2026Taming Various Privilege Escalation in LLM-Based Agent Systems: A Mandatory Access Control Framework17 Jan
  5. 2026Towards Verifiably Safe Tool Use for LLM Agents12 Jan
  6. 2025Fault-Tolerant Sandboxing for AI Coding Agents: A Transactional Approach to Safe Autonomous Execution14 Dec
  7. 2025Secure and Efficient Access Control for Computer-Use Agents via Context Space26 Sep
  8. 2025LLMZ+: Contextual Prompt Whitelist Principles for Agentic LLMs23 Sep
  9. 2025AgentArmor: Enforcing Program Analysis on Agent Runtime Trace to Defend Against Prompt Injection2 Aug
  10. 2025DRIFT: Dynamic Rule-Based Defense with Injection Isolation for Securing LLM Agents13 Jun
  11. 2025SAFEFLOW: A Principled Protocol for Trustworthy and Transactional Autonomous Agent Systems9 Jun
  12. 2025Defeating Prompt Injections by Design24 Mar
7 more
  1. 2025AgentSpec: Customizable Runtime Enforcement for Safe and Reliable LLM Agents24 Mar
  2. 2025Prompt Flow Integrity to Prevent Privilege Escalation in LLM Agents17 Mar
  3. 2025Adaptive Attacks Break Defenses Against Indirect Prompt Injection Attacks on LLM Agents27 Feb
  4. 2025AGrail: A Lifelong Agent Guardrail with Effective and Adaptive Safety Detection17 Feb
  5. 2025RTBAS: Defending LLM Agents Against Prompt Injection and Privacy Leakage13 Feb
  6. 2024The Task Shield: Enforcing Task Alignment to Defend Against Indirect Prompt Injection in LLM Agents21 Dec
  7. 2024IsolateGPT: An Execution Isolation Architecture for LLM-Based Agentic Systems8 Mar

Other references · 19

  1. 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,393
  2. 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,355
  3. 2024Understanding the planning of LLM agents: A surveycited by 414
  4. 2023NeMo Guardrails: A Toolkit for Controllable and Safe LLM Applications with Programmable Railscited by 347
  5. 2025Securing AI Agents with Information-Flow Controlcited by 108
  6. 2024System-Level Defense against Indirect Prompt Injection Attacks: An Information Flow Control Perspectivecited by 91
  7. 2024An Empirical Study of LLM-as-a-Judge for LLM Evaluation: Fine-tuned Judge Model is not a General Substitute for GPT-4cited by 90
  8. 2025Contextual Agent Security: A Policy for Every Purposecited by 55
  9. 2025ACE: A Security Architecture for LLM-Integrated App Systemscited by 52
  10. 2025What Prompts Don't Say: Understanding and Managing Underspecification in LLM Promptscited by 32
  11. 2025Ambig-SWE: Interactive Agents to Overcome Underspecificity in Software Engineeringcited by 28
  12. 2026Don't Let the Claw Grip Your Hand: A Security Analysis and Defense Framework for OpenClawcited by 26
7 more
  1. 2024Permissive Information-Flow Analysis for Large Language Modelscited by 22
  2. 2025Towards Automating Data Access Permissions in AI Agentscited by 20
  3. 2024FATH: Authentication-based Test-time Defense against Indirect Prompt Injection Attackscited by 19
  4. 2024Contextualized Evaluations: Judging Language Model Responses to Underspecified Queriescited by 12
  5. 2026AgentGuardian: Learning Access Control Policies to Govern AI Agent Behaviorcited by 9
  6. 2025On the Regulatory Potential of User Interfaces for AI Agent Governancecited by 3
  7. 2023Stuck in the Permissions With You: Developer & End-User Perspectives on App Permissions & Their Privacy Ramificationscited by 2

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 8.

8 papers.

Similar, not cited

Close in wording, no citation either way

  1. 2026LLM-Based Agents for Software and Systems Security: Approaches, Applications, and Assessment
  2. 2026Connecting the Dots in Agentic AI Security: A Cross-Dimensional Threat Taxonomy, Evaluation Maturity, and Open Challenges
  3. 2025TRiSM for Agentic AI: A Review of Trust, Risk, and Security Management in LLM-based Agentic Multi-Agent Systems
  4. 2026From Human-Human Collaboration to Human-Agent Collaboration: A Vision, Design Philosophy, and an Empirical Framework for Achieving Successful Partnerships Between Humans and LLM Agents
  5. 2026Delegation Without Trust: An Empirical Gap Analysis of Identity, Authorization, and Runtime Governance in Multi-Agent LLM Systems
  6. 2026Toward Secure LLM Agents: Threat Surfaces, Attacks, Defenses, and Evaluation
  7. 2026How Well Can LLM Agents Simulate End-User Security and Privacy Attitudes and Behaviors?
  8. 2025Seven Security Challenges in Cross-domain Multi-agent LLM Systems