mlsanityData to 14 Oct 2026

Build the web for agents, not agents for the web

arXiv 2506.10953 ↗First posted 12 Jun 2025cs.LG cs.CLcited by 18

On the map: LLM agents and tool use → Web, search, and research agents

Abstract (arXiv)

Recent advancements in Large Language Models (LLMs) and multimodal counterparts have spurred significant interest in developing web agents -- AI systems capable of autonomously navigating and completing tasks within web environments. While holding tremendous promise for automating complex web interactions, current approaches face substantial challenges due to the fundamental mismatch between human-designed interfaces and LLM capabilities. Current methods struggle with the inherent complexity of web inputs, whether processing massive DOM trees, relying on screenshots augmented with additional information, or bypassing the user interface entirely through API interactions. This position paper advocates for a paradigm shift in web agent research: rather than forcing web agents to adapt to interfaces designed for humans, we should develop a new interaction paradigm specifically optimized for agentic capabilities. To this end, we introduce the concept of an Agentic Web Interface (AWI), an interface specifically designed for agents to navigate a website. We establish six guiding principles for AWI design, emphasizing safety, efficiency, and standardization, to account for the interests of all primary stakeholders. This reframing aims to overcome fundamental limitations of existing interfaces, paving the way for more efficient, reliable, and transparent web agent design, which will be a collaborative effort involving the broader ML community.

Key facts. Cited by 18 papers in the map (data through 14 Oct 2026). Placed in Web, search, and research agents, one of 16 directions in LLM agents and tool use. Builds on 50 of its 64 references that are in the map. First posted on arXiv 12 Jun 2025.

Builds on

50 of 64 references.

Closest ancestors · 15

Same direction, newest first

  1. 2025SkillWeaver: Web Agents can Self-Improve by Discovering and Honing Skills9 Apr
  2. 2025An Illusion of Progress? Assessing the Current State of Web Agents2 Apr
  3. 2025InSTA: Towards Internet-Scale Training For Agents10 Feb
  4. 2024Is Your LLM Secretly a World Model of the Internet? Model-Based Planning for Web Agents10 Nov
  5. 2024From Context to Action: Analysis of the Impact of State Representation and Context on the Generalization of Multi-Turn Web Navigation Agents31 Oct
  6. 2024Beyond Browsing: API-Based Web Agents21 Oct
  7. 2024NNetNav: Unsupervised Learning of Browser Agents Through Environment Interaction in the Wild3 Oct
  8. 2024WebCanvas: Benchmarking Web Agents in Online Environments18 Jun
  9. 2024Large Language Models Can Self-Improve At Web Agent Tasks30 May
  10. 2024AutoWebGLM: A Large Language Model-based Web Navigating Agent4 Apr
  11. 2024WorkArena: How Capable Are Web Agents at Solving Common Knowledge Work Tasks?12 Mar
  12. 2024WebVoyager: Building an End-to-End Web Agent with Large Multimodal Models25 Jan
3 more
  1. 2024GPT-4V(ision) is a Generalist Web Agent, if Grounded3 Jan
  2. 2023WebArena: A Realistic Web Environment for Building Autonomous Agents25 Jul
  3. 2023Mind2Web: Towards a Generalist Agent for the Web9 Jun

Other references · 35

  1. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,297
  2. 2023Reflexion: Language Agents with Verbal Reinforcement Learningcited by 4,276
  3. 2023Toolformer: Language Models Can Teach Themselves to Use Toolscited by 4,182
  4. 2020Dense Passage Retrieval for Open-Domain Question Answeringcited by 3,824
  5. 2020REALM: Retrieval-Augmented Language Model Pre-Trainingcited by 2,141
  6. 2024SWE-agent: Agent-Computer Interfaces Enable Automated Software Engineeringcited by 1,737
  7. 2024OSWorld: Benchmarking Multimodal Agents for Open-Ended Tasks in Real Computer Environmentscited by 1,195
  8. 2019Latent Retrieval for Weakly Supervised Open Domain Question Answeringcited by 695
  9. 2025UI-TARS: Pioneering Automated GUI Interaction with Native Agentscited by 610
  10. 2024AndroidWorld: A Dynamic Benchmarking Environment for Autonomous Agentscited by 468
  11. 2024AgentHarm: A Benchmark for Measuring Harmfulness of LLM Agentscited by 437
  12. 2024LLM2Vec: Large Language Models Are Secretly Powerful Text Encoderscited by 404
23 more
  1. 2024Navigating the Digital World as Humans Do: Universal Visual Grounding for GUI Agentscited by 382
  2. 2023A Real-World WebAgent with Planning, Long Context Understanding, and Program Synthesiscited by 353
  3. 2023Android in the Wild: A Large-Scale Dataset for Android Device Controlcited by 319
  4. 2018Reinforcement Learning on Web Interfaces Using Workflow-Guided Explorationcited by 284
  5. 2024On the Effects of Data Scale on UI Control Agentscited by 194
  6. 2024WebLINX: Real-World Website Navigation with Multi-Turn Dialoguecited by 180
  7. 2023ADaPT: As-Needed Decomposition and Planning with Language Modelscited by 177
  8. 2023Multimodal Web Navigation with Instruction-Finetuned Foundation Modelscited by 160
  9. 2024Tree Search for Language Model Agentscited by 151
  10. 2024ST-WebAgentBench: A Benchmark for Evaluating Safety and Trustworthiness in Web Agentscited by 133
  11. 2024VisualAgentBench: Towards Large Multimodal Models as Visual Foundation Agentscited by 103
  12. 2025SafeArena: Evaluating the Safety of Autonomous Web Agentscited by 102
  13. 2024VisualWebBench: How Far Have Multimodal LLMs Evolved in Web Page Understanding and Grounding?cited by 100
  14. 2024WorkArena++: Towards Compositional Planning and Reasoning-based Common Knowledge Work Taskscited by 90
  15. 2023From Pixels to UI Actions: Learning to Follow Instructions via Graphical User Interfacescited by 68
  16. 2023SteP: Stacked LLM Policies for Web Actionscited by 66
  17. 2024Large Language Models Empowered Personalized Web Agentscited by 66
  18. 2023SummIt: Iterative Text Summarization via ChatGPTcited by 54
  19. 2025RedTeamCUA: Realistic Adversarial Testing of Computer-Use Agents in Hybrid Web-OS Environmentscited by 46
  20. 2024VideoWebArena: Evaluating Long Context Multimodal Agents with Video Understanding Web Taskscited by 43
  21. 2024BAGEL: Bootstrapping Agents by Guiding Exploration with Languagecited by 34
  22. 2025DoomArena: A framework for Testing AI Agents Against Evolving Security Threatscited by 32
  23. 2024How Culture Shapes What People Want From AIcited by 23

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 7; 2026: 11.

18 papers.

Sort:

By area:

  1. 2025From Prompt Injections to Protocol Exploits: Threats in LLM-Powered AI Agents Workflowscited by 58
  2. 2025Agentic Web: Weaving the Next Web with AI Agentscited by 45
  3. 2025WALT: Web Agents that Learn Toolscited by 16
  4. 2026ACE-Router: Generalizing History-Aware Routing from MCP Tools to the Agent Webcited by 7
  5. 2026How do Visual Attributes Influence Web Agents? A Comprehensive Evaluation of User Interface Design Factorscited by 4
  6. 2026AgentWebBench: Benchmarking Multi-Agent Coordination in Agentic Webcited by 4
  7. 2025Interaction-Driven Browsing: A Human-in-the-Loop Conceptual Framework Informed by Human Web Browsing for Browser-Using Agentscited by 3
  8. 2025WebRouter: Query-specific Router via Variational Information Bottleneck for Cost-sensitive Web Agentcited by 2
  9. 2025Affordance Representation and Recognition for Autonomous Agentscited by 2
  10. 2025Reliable agent engineering should integrate machine-compatible organizational principlescited by 2
  11. 2026DeepXiv-SDK: An Agentic Data Interface for Scientific Literaturecited by 2
  12. 2026AgentBeats: Agentifying Agent Assessment for Openness, Standardization, and Reproducibilitycited by 2
6 more
  1. 2026WebSP-Eval: Evaluating Web Agents on Website Security and Privacy Taskscited by 1
  2. 2026CI4A: Semantic Component Interfaces for Agents Empowering Web Automationcited by 0
  3. 2026Internal APIs Are All You Need: Shadow APIs, Shared Discovery, and the Case Against Browser-First Agent Architecturescited by 0
  4. 2026SW-A2-Bench: Benchmarking Autonomous Software Agent Generation for Agentic Webcited by 0
  5. 2026AnyAct: Universal Action for Self-Evolving Agentscited by 0
  6. 2026Get In, Decide, Prove: A Merchant-Side Readiness Benchmark for Agentic Commercecited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026Webhell: A Unified Interface for Agentic Web Access
  2. 2024Agent-E: From Autonomous Web Navigation to Foundational Design Principles in Agentic Systems
  3. 2025Building the Web for Agents: A Declarative Framework for Agent-Web Interaction
  4. 2026Web Agents Should Use Typed Actions Instead of Click-Based Browsing
  5. 2024AgentOccam: A Simple Yet Strong Baseline for LLM-Based Web Agents
  6. 2025AgentA/B: Automated and Scalable Web A/BTesting with Interactive LLM Agents
  7. 2026WebChallenger: A Reliable and Efficient Generalist Web Agent
  8. 2024MMInA: Benchmarking Multihop Multimodal Internet Agents