# AC4A: Access Control for Agents

arXiv: https://arxiv.org/abs/2603.20933 · HTML: https://mlsanity.com/paper/2603.20933/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Placed in Agent threats, guardrails and governance, one of 16 directions in LLM agents and tool use.
- Builds on 14 of its 31 references that are in the map.
- First posted on arXiv 21 Mar 2026.

## Placed in

- LLM agents and tool use / [Agent threats, guardrails and governance](https://mlsanity.com/agents/agent-threats-guardrails-and-governance/)

## Builds on (most cited first, 14 of 14)

- [AutoGen: Enabling Next-Gen LLM Applications via Multi-Agent Conversation](https://mlsanity.com/paper/2308.08155/) (2023)
- [A Survey on Hallucination in Large Language Models: Principles, Taxonomy, Challenges, and Open Questions](https://mlsanity.com/paper/2311.05232/) (2023)
- [OSWorld: Benchmarking Multimodal Agents for Open-Ended Tasks in Real Computer Environments](https://mlsanity.com/paper/2404.07972/) (2024)
- [Defeating Prompt Injections by Design](https://mlsanity.com/paper/2503.18813/) (2025)
- [PoisonedRAG: Knowledge Corruption Attacks to Retrieval-Augmented Generation of Large Language Models](https://mlsanity.com/paper/2402.07867/) (2024)
- [Progent: Securing AI Agents with Privilege Control](https://mlsanity.com/paper/2504.11703/) (2025)
- [IsolateGPT: An Execution Isolation Architecture for LLM-Based Agentic Systems](https://mlsanity.com/paper/2403.04960/) (2024)
- [Securing AI Agents with Information-Flow Control](https://mlsanity.com/paper/2505.23643/) (2025)
- [Automatic and Universal Prompt Injection Attacks against Large Language Models](https://mlsanity.com/paper/2403.04957/) (2024)
- [System-Level Defense against Indirect Prompt Injection Attacks: An Information Flow Control Perspective](https://mlsanity.com/paper/2409.19091/) (2024)
- [Contextual Agent Security: A Policy for Every Purpose](https://mlsanity.com/paper/2501.17070/) (2025)
- [Traceback of Poisoning Attacks to Retrieval-Augmented Generation](https://mlsanity.com/paper/2504.21668/) (2025)
- [Cognitive Overload Attack:Prompt Injection for Long Context](https://mlsanity.com/paper/2410.11272/) (2024)
- [What Really Matters in Many-Shot Attacks? An Empirical Study of Long-Context Vulnerabilities in LLMs](https://mlsanity.com/paper/2505.19773/) (2025)

## Built on by (most cited first, 5 of 5)

- [Janus: a Playground for User-Involved Agentic Permission Management](https://mlsanity.com/paper/2607.01510/) (2026)
- [Towards an Agent Operating System - Lessons from Classical and Cloud OS](https://mlsanity.com/paper/2607.25076/) (2026)
- [Delegation Without Trust: An Empirical Gap Analysis of Identity, Authorization, and Runtime Governance in Multi-Agent LLM Systems](https://mlsanity.com/paper/2609.00267/) (2026)
- [Engineering Robustness into Personal Agents with the AI Workflow Store](https://mlsanity.com/paper/2605.10907/) (2026)
- [Ajar: Measuring Open Privilege in Agent Defenses](https://mlsanity.com/paper/2609.26900/) (2026)

## Abstract (arXiv)

Large Language Model (LLM) agents combine the chat interaction capabilities of LLMs with the power to interact with external tools and APIs. This enables them to perform complex tasks and act autonomously to achieve user goals. However, current agent systems operate on an all-or-nothing basis: an agent either has full access to an API's capabilities and a web page's content, or it has no access at all. This coarse-grained approach forces users to trust agents with more capabilities than they actually need for a given task. In this paper, we introduce AC4A, an access control framework for agents. As agents become more capable and autonomous, users need a way to limit what APIs or portions of web pages these agents can access, eliminating the need to trust them with everything an API or web page allows. Our goal with AC4A is to provide a framework for defining permissions that lets agents access only the resources they are authorized to access. AC4A works across both API-based and browser-based agents. It does not prescribe what permissions should be, but offers a flexible way to define and enforce them, making it practical for real-world systems. AC4A works by creating permissions granting access to resources, drawing inspiration from established access control frameworks like the one for the Unix file system. Applications define their resources as hierarchies and provide a way to compute the necessary permissions at runtime needed for successful resource access. We demonstrate the usefulness of AC4A in enforcing permissions over real-world APIs and web pages through case studies. The source code of AC4A is available at https://github.com/reSHARMA/AC4A