A Vision for Access Control in LLM-based Agent Systems
On the map: LLM agents and tool use → Agent threats, guardrails and governance
Abstract (arXiv)
The autonomy and contextual complexity of LLM-based agents render traditional access control (AC) mechanisms insufficient. Static, rule-based systems designed for predictable environments are fundamentally ill-equipped to manage the dynamic information flows inherent in agentic interactions. This position paper argues for a paradigm shift from binary access control to a more sophisticated model of information governance, positing that the core challenge is not merely about permission, but about governing the flow of information. We introduce Agent Access Control (AAC), a novel framework that reframes AC as a dynamic, context-aware process of information flow governance. AAC operates on two core modules: (1) multi-dimensional contextual evaluation, which assesses not just identity but also relationships, scenarios, and norms; and (2) adaptive response formulation, which moves beyond simple allow/deny decisions to shape information through redaction, summarization, and paraphrasing. This vision, powered by a dedicated AC reasoning engine, aims to bridge the gap between human-like nuanced judgment and scalable Al safety, proposing a new conceptual lens for future research in trustworthy agent design.
Key facts. Cited by 14 papers in the map (data through 7 Oct 2026). Placed in Agent threats, guardrails and governance, one of 16 directions in LLM agents and tool use. Builds on 25 of its 34 references that are in the map. First posted on arXiv 13 Oct 2025.
Builds on
25 of 34 references.
Closest ancestors · 6
Same direction, newest first
- 2025A-MemGuard: A Proactive Defense Framework for LLM-Based Agent Memory29 Sep
- 2025AgentAuditor: Human-Level Safety and Security Evaluation for LLM Agents31 May
- 2025MedSentry: Understanding and Mitigating Safety Risks in Medical LLM Multi-Agent Systems27 May
- 2025A Comprehensive Survey in LLM(-Agent) Full Stack Safety: Data, Training and Deployment22 Apr
- 2025A Survey on Trustworthy LLM Agents: Threats and Countermeasures12 Mar
- 2024From Allies to Adversaries: Manipulating LLM Tool-Calling through Adversarial Injection13 Dec
Other references · 19
- 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,355
- 2023Universal and Transferable Adversarial Attacks on Aligned Language Modelscited by 2,914
- 2023Voyager: An Open-Ended Embodied Agent with Large Language Modelscited by 2,081
- 2023Not what you've signed up for: Compromising Real-World LLM-Integrated Applications with Indirect Prompt Injectioncited by 1,339
- 2016Man is to Computer Programmer as Woman is to Homemaker? Debiasing Word Embeddingscited by 1,304
- 2021A General Language Assistant as a Laboratory for Alignmentcited by 1,023
- 2016Semantics derived automatically from language corpora contain human-like biasescited by 894
- 2023Improving Language Model Negotiation with Self-Play and In-Context Learning from AI Feedbackcited by 202
- 2024Agent AI: Surveying the Horizons of Multimodal Interactioncited by 167
- 2025Advances and Challenges in Foundation Agents: From Brain-Inspired Intelligence to Evolutionary, Collaborative, and Safe Systemscited by 125
- 2017Ethical Challenges in Data-Driven Dialogue Systemscited by 86
- 2025A Survey of LLM-based Agents in Medicine: How far are we from Baymax?cited by 84
7 more
- 2024SafeGen: Mitigating Sexually Explicit Content Generation in Text-to-Image Modelscited by 64
- 2023ReEval: Automatic Hallucination Evaluation for Retrieval-Augmented Large Language Models via Transferable Adversarial Attackscited by 28
- 2021Am I Me or You? State-of-the-Art Dialogue Models Cannot Maintain an Identitycited by 22
- 2025AudioTrust: Benchmarking the Multifaceted Trustworthiness of Audio Large Language Modelscited by 12
- 2025The Eye of Sherlock Holmes: Uncovering User Private Attribute Profiling via Vision-Language Model Agentic Frameworkcited by 9
- 2025The Man Behind the Sound: Demystifying Audio Private Attribute Profiling via Multimodal Large Language Model Agentscited by 5
- 2018Procedurally Provisioned Access Control for Robotic Systemscited by 2
Built on by
Citing papers per year: 2025: 2; 2026: 12.
14 papers.
Sort:
By area:
- 2026A Survey on Long-Term Memory Security in LLM Agents: Attacks, Defenses, and Governance Across the Memory Lifecyclecited by 21
- 2026Self-Improvements in Modern Agentic Systems: A Surveycited by 17
- 2025SoK: Trust-Authorization Mismatch in LLM Agent Interactionscited by 11
- 2025Security Analysis of Agentic AI Communication Protocols: A Comparative Evaluationcited by 6
- 2026Do Coding Agents Understand Least-Privilege Authorization?cited by 4
- 2026You Told Me to Do It: Measuring Instructional Text-induced Private Data Leakage in LLM Agentscited by 3
- 2026PlanTwin: Privacy-Preserving Planning Abstractions for Cloud-Assisted LLM Agentscited by 2
- 2026VERA: Authority-Preserving Edge Revocation for Federated AI-Agent Workflowscited by 2
- 2026Local Is Not a Sufficient Privacy Boundary: Governing OS-Integrated On-Device AIcited by 1
- 2026Agents That Know Too Much: A Data-Centric Survey of Privacy in LLM Agentscited by 0
- 2026The Balkanization of Execution-Security Research for AI Coding Agents: Isolation, Access Control, and Time-of-Check-to-Time-of-Use Vulnerabilitiescited by 0
- 2026MemArena: An Ego-Centric Benchmark for On-Device Agentic Personal Memory Assistants at Scalecited by 0
- 2026OverAct: Measuring and Mitigating Proactive Over-Authorization in LLM Tool-Calling Agents1 Oct 2026
- 2026When the Agent Becomes the Kernel: A Systematization of Security on the Path to AI-Native Operating Systems20 Sep 2026
- 2026VERA: Authority-Preserving Edge Revocation for Federated AI-Agent Workflows30 Aug 2026
- 2026Self-Improvements in Modern Agentic Systems: A Survey14 Jul 2026
- 2026The Balkanization of Execution-Security Research for AI Coding Agents: Isolation, Access Control, and Time-of-Check-to-Time-of-Use Vulnerabilities7 Jul 2026
- 2026Agents That Know Too Much: A Data-Centric Survey of Privacy in LLM Agents25 Jun 2026
- 2026Local Is Not a Sufficient Privacy Boundary: Governing OS-Integrated On-Device AI8 Jun 2026
- 2026MemArena: An Ego-Centric Benchmark for On-Device Agentic Personal Memory Assistants at Scale20 May 2026
- 2026Do Coding Agents Understand Least-Privilege Authorization?14 May 2026
- 2026A Survey on Long-Term Memory Security in LLM Agents: Attacks, Defenses, and Governance Across the Memory Lifecycle17 Apr 2026
- 2026PlanTwin: Privacy-Preserving Planning Abstractions for Cloud-Assisted LLM Agents19 Mar 2026
- 2026You Told Me to Do It: Measuring Instructional Text-induced Private Data Leakage in LLM Agents12 Mar 2026
- 2026A Survey on Long-Term Memory Security in LLM Agents: Attacks, Defenses, and Governance Across the Memory Lifecyclecited by 21
- 2026Self-Improvements in Modern Agentic Systems: A Surveycited by 17
- 2025SoK: Trust-Authorization Mismatch in LLM Agent Interactionscited by 11
- 2025Security Analysis of Agentic AI Communication Protocols: A Comparative Evaluationcited by 6
- 2026Do Coding Agents Understand Least-Privilege Authorization?cited by 4
- 2026You Told Me to Do It: Measuring Instructional Text-induced Private Data Leakage in LLM Agentscited by 3
- 2026PlanTwin: Privacy-Preserving Planning Abstractions for Cloud-Assisted LLM Agentscited by 2
- 2026Agents That Know Too Much: A Data-Centric Survey of Privacy in LLM Agentscited by 0
- 2026The Balkanization of Execution-Security Research for AI Coding Agents: Isolation, Access Control, and Time-of-Check-to-Time-of-Use Vulnerabilitiescited by 0
- 2026MemArena: An Ego-Centric Benchmark for On-Device Agentic Personal Memory Assistants at Scalecited by 0
- 2026When the Agent Becomes the Kernel: A Systematization of Security on the Path to AI-Native Operating Systemscited by 0
- 2026OverAct: Measuring and Mitigating Proactive Over-Authorization in LLM Tool-Calling Agentscited by 0
- 2026OverAct: Measuring and Mitigating Proactive Over-Authorization in LLM Tool-Calling Agents1 Oct 2026
- 2026When the Agent Becomes the Kernel: A Systematization of Security on the Path to AI-Native Operating Systems20 Sep 2026
- 2026Self-Improvements in Modern Agentic Systems: A Survey14 Jul 2026
- 2026The Balkanization of Execution-Security Research for AI Coding Agents: Isolation, Access Control, and Time-of-Check-to-Time-of-Use Vulnerabilities7 Jul 2026
- 2026Agents That Know Too Much: A Data-Centric Survey of Privacy in LLM Agents25 Jun 2026
- 2026MemArena: An Ego-Centric Benchmark for On-Device Agentic Personal Memory Assistants at Scale20 May 2026
- 2026Do Coding Agents Understand Least-Privilege Authorization?14 May 2026
- 2026A Survey on Long-Term Memory Security in LLM Agents: Attacks, Defenses, and Governance Across the Memory Lifecycle17 Apr 2026
- 2026PlanTwin: Privacy-Preserving Planning Abstractions for Cloud-Assisted LLM Agents19 Mar 2026
- 2026You Told Me to Do It: Measuring Instructional Text-induced Private Data Leakage in LLM Agents12 Mar 2026
- 2025SoK: Trust-Authorization Mismatch in LLM Agent Interactions7 Dec 2025
- 2025Security Analysis of Agentic AI Communication Protocols: A Comparative Evaluation5 Nov 2025
Similar, not cited
Close in wording, no citation either way
- 2025Uncertainty-Aware, Risk-Adaptive Access Control for Agentic Systems using an LLM-Judged TBAC Model
- 2026Janus: a Playground for User-Involved Agentic Permission Management
- 2025LLM Agents Should Employ Security Principles
- 2026Hybrid Inspection and Task-Based Access Control in Zero-Trust Agentic AI
- 2024Integrated Design and Governance of Agentic AI Systems through Adaptive Information Modulation
- 2025Secure and Efficient Access Control for Computer-Use Agents via Context Space
- 2026Taming Various Privilege Escalation in LLM-Based Agent Systems: A Mandatory Access Control Framework
- 2026Overlaying Governance: A Compositional Authorization Framework for Delegation and Scope in Agentic AI