# Permission Manifests for Web Agents

arXiv: https://arxiv.org/abs/2601.02371 · HTML: https://mlsanity.com/paper/2601.02371/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Placed in Web, search, and research agents, one of 16 directions in LLM agents and tool use.
- Builds on 13 of its 43 references that are in the map.
- First posted on arXiv 7 Dec 2025.

## Placed in

- LLM agents and tool use / [Web, search, and research agents](https://mlsanity.com/agents/web-search-and-research-agents/)

## Builds on (most cited first, 13 of 13)

- [WebArena: A Realistic Web Environment for Building Autonomous Agents](https://mlsanity.com/paper/2307.13854/) (2023)
- [WebGPT: Browser-assisted question-answering with human feedback](https://mlsanity.com/paper/2112.09332/) (2021)
- [WorkArena: How Capable Are Web Agents at Solving Common Knowledge Work Tasks?](https://mlsanity.com/paper/2403.07718/) (2024)
- [ResearchAgent: Iterative Research Idea Generation over Scientific Literature with Large Language Models](https://mlsanity.com/paper/2404.07738/) (2024)
- [Plan-and-Act: Improving Planning of Agents for Long-Horizon Tasks](https://mlsanity.com/paper/2503.09572/) (2025)
- [WorkArena++: Towards Compositional Planning and Reasoning-based Common Knowledge Work Tasks](https://mlsanity.com/paper/2407.05291/) (2024)
- [Authenticated Delegation and Authorized AI Agents](https://mlsanity.com/paper/2501.09674/) (2025)
- [WebInject: Prompt Injection Attack to Web Agents](https://mlsanity.com/paper/2505.11717/) (2025)
- [OSCAR: Operating System Control via State-Aware Reasoning and Re-Planning](https://mlsanity.com/paper/2410.18963/) (2024)
- [Enabling Uniform Computer Interaction Experience for Blind Users through Large Language Models](https://mlsanity.com/paper/2407.19537/) (2024)
- [LLaSA: Large Language and E-Commerce Shopping Assistant](https://mlsanity.com/paper/2408.02006/) (2024)
- [Scrapers selectively respect robots.txt directives: evidence from a large-scale empirical study](https://mlsanity.com/paper/2505.21733/) (2025)
- [Web Scraping for Research: Legal, Ethical, Institutional, and Scientific Considerations](https://mlsanity.com/paper/2410.23432/) (2024)

## Built on by (most cited first, 5 of 5)

- [From Chatbot to Digital Colleague: The Paradigm Shift Toward Persistent Autonomous AI](https://mlsanity.com/paper/2606.14502/) (2026)
- [Will the Agent Recuse, and Will It Stop? Measuring LLM-Agent Compliance with In-Band Governance Signals at the Access Door and Mid-Flight](https://mlsanity.com/paper/2606.06460/) (2026)
- [Securing Computer-Use Agents: A Unified Architecture-Lifecycle Framework for Deployment-Grounded Reliability](https://mlsanity.com/paper/2605.07110/) (2026)
- [The Agentic Web Requires New Normative Infrastructure](https://mlsanity.com/paper/2606.10711/) (2026)
- [Get In, Decide, Prove: A Merchant-Side Readiness Benchmark for Agentic Commerce](https://mlsanity.com/paper/2610.17378/) (2026)

## Abstract (arXiv)

The rise of Large Language Model (LLM)-based web agents represents a significant shift in automated interactions with the web. Unlike traditional crawlers that follow simple conventions, such as robots.txt, modern agents engage with websites in sophisticated ways: navigating complex interfaces, extracting structured information, and completing end-to-end tasks. Existing governance mechanisms were not designed for these capabilities. Without a way to specify what interactions are and are not allowed, website owners increasingly rely on blanket blocking and CAPTCHAs, which undermine beneficial applications such as efficient automation, convenient use of e-commerce services, and accessibility tools. We introduce agent-permissions.json, a robots.txt-style lightweight manifest where websites specify allowed interactions, complemented by API references where available. This framework provides a low-friction coordination mechanism: website owners only need to write a simple JSON file, while agents can easily parse and automatically implement the manifest's provisions. Website owners can then focus on blocking non-compliant agents, rather than agents as a whole. By extending the spirit of robots.txt to the era of LLM-mediated interaction, and complementing data use initiatives such as AIPref, the manifest establishes a compliance framework that enables beneficial agent interactions while respecting site owners' preferences.