mlsanityData to 7 Oct 2026

Beyond Browsing: API-Based Web Agents

arXiv 2410.16464 ↗First posted 21 Oct 2024cs.CL cs.MAcited by 77

On the map: LLM agents and tool use → Web, search, and research agents

Abstract (arXiv)

Web browsers are a portal to the internet, where much of human activity is undertaken. Thus, there has been significant research work in AI agents that interact with the internet through web browsing. However, there is also another interface designed specifically for machine interaction with online content: application programming interfaces (APIs). In this paper we ask -- what if we were to take tasks traditionally tackled by Browsing Agents, and give AI agents access to APIs? To do so, we propose two varieties of agents: (1) an API-calling agent that attempts to perform online tasks through APIs only, similar to traditional coding agents, and (2) a Hybrid Agent that can interact with online data through both web browsing and APIs. In experiments on WebArena, a widely-used and realistic benchmark for web navigation tasks, we find that API-Based Agents outperform web Browsing Agents. Hybrid Agents out-perform both others nearly uniformly across tasks, resulting in a more than 24.0% absolute improvement over web browsing alone, achieving a success rate of 38.9%, the SOTA performance among task-agnostic agents. These results strongly suggest that when APIs are available, they present an attractive alternative to relying on web browsing alone.

Key facts. Cited by 77 papers in the map (data through 7 Oct 2026). Placed in Web, search, and research agents, one of 16 directions in LLM agents and tool use. Builds on 43 of its 66 references that are in the map. First posted on arXiv 21 Oct 2024.

Builds on

43 of 66 references.

Closest ancestors · 9

Same direction, newest first

  1. 2024Agent Workflow Memory11 Sep
  2. 2024WILBUR: Adaptive In-Context Learning for Robust and Accurate Web Agents8 Apr
  3. 2024AutoWebGLM: A Large Language Model-based Web Navigating Agent4 Apr
  4. 2024WorkArena: How Capable Are Web Agents at Solving Common Knowledge Work Tasks?12 Mar
  5. 2024WebVoyager: Building an End-to-End Web Agent with Large Multimodal Models25 Jan
  6. 2024GPT-4V(ision) is a Generalist Web Agent, if Grounded3 Jan
  7. 2023WebArena: A Realistic Web Environment for Building Autonomous Agents25 Jul
  8. 2023Mind2Web: Towards a Generalist Agent for the Web9 Jun
  9. 2022WebShop: Towards Scalable Real-World Web Interaction with Grounded Language Agents4 Jul

Other references · 34

  1. 2023ToolLLM: Facilitating Large Language Models to Master 16000+ Real-world APIscited by 1,880
  2. 2023Gorilla: Large Language Model Connected with Massive APIscited by 1,302
  3. 2023AgentBench: Evaluating LLMs as Agentscited by 1,211
  4. 2024OSWorld: Benchmarking Multimodal Agents for Open-Ended Tasks in Real Computer Environmentscited by 1,134
  5. 2024OpenHands: An Open Platform for AI Software Developers as Generalist Agentscited by 956
  6. 2023CogAgent: A Visual Language Model for GUI Agentscited by 692
  7. 2024Executable Code Actions Elicit Better LLM Agentscited by 537
  8. 2023AppAgent: Multimodal Agents as Smartphone Userscited by 407
  9. 2024AppWorld: A Controllable World of Apps and People for Benchmarking Interactive Coding Agentscited by 359
  10. 2023A Real-World WebAgent with Planning, Long Context Understanding, and Program Synthesiscited by 350
  11. 2023Android in the Wild: A Large-Scale Dataset for Android Device Controlcited by 314
  12. 2018Reinforcement Learning on Web Interfaces Using Workflow-Guided Explorationcited by 282
22 more
  1. 2024CodeAgent: Enhancing Code Generation with Tool-Integrated Agent Systems for Real-World Repo-level Coding Challengescited by 278
  2. 2024Trial and Error: Exploration-Based Trajectory Optimization for LLM Agentscited by 194
  3. 2020Mapping Natural Language Instructions to Mobile UI Action Sequencescited by 180
  4. 2024WebLINX: Real-World Website Navigation with Multi-Turn Dialoguecited by 177
  5. 2024Ferret-UI: Grounded Mobile UI Understanding with Multimodal LLMscited by 173
  6. 2024EASYTOOL: Enhancing LLM-based Agents with Concise Tool Instructioncited by 153
  7. 2024Tree Search for Language Model Agentscited by 148
  8. 2024Autonomous Evaluation and Refinement of Digital Agentscited by 136
  9. 2024AnyTool: Self-Reflective, Hierarchical Agents for Large-Scale API Callscited by 121
  10. 2024VisualAgentBench: Towards Large Multimodal Models as Visual Foundation Agentscited by 101
  11. 2024VisualWebBench: How Far Have Multimodal LLMs Evolved in Web Page Understanding and Grounding?cited by 98
  12. 2024Agent-FLAN: Designing Data and Methods of Effective Agent Tuning for Large Language Modelscited by 97
  13. 2024PersonaGym: Evaluating Persona Agents and LLMscited by 89
  14. 2024Internet of Agents: Weaving a Web of Heterogeneous Agents for Collaborative Intelligencecited by 80
  15. 2024Visibility into AI Agentscited by 77
  16. 2024AutoGuide: Automated Generation and Selection of Context-Aware Guidelines for Large Language Model Agentscited by 69
  17. 2023SteP: Stacked LLM Policies for Web Actionscited by 66
  18. 2022A Dataset for Interactive Vision-Language Navigation with Unknown Command Feasibilitycited by 61
  19. 2024What Are Tools Anyway? A Survey from the Language Model Perspectivecited by 53
  20. 2021Grounding Open-Domain Instructions to Automate Web Support Taskscited by 43
  21. 2024Middleware for LLMs: Tools Are Instrumental for Language Agents in Complex Environmentscited by 39
  22. 2024ShortcutsBench: A Large-Scale Real-world Benchmark for API-based Agentscited by 36

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 4; 2025: 41; 2026: 32.

77 papers.

Sort:

By area:

  1. 2024TheAgentCompany: Benchmarking LLM Agents on Consequential Real World Taskscited by 306
  2. 2025Plan-and-Act: Improving Planning of Agents for Long-Horizon Taskscited by 190
  3. 2025SkillWeaver: Web Agents can Self-Improve by Discovering and Honing Skillscited by 174
  4. 2025WebWalker: Benchmarking LLMs in Web Traversalcited by 171
  5. 2025AI Agents vs. Agentic AI: A Conceptual Taxonomy, Applications and Challengescited by 167
  6. 2024Large Language Model-Brained GUI Agents: A Surveycited by 160
  7. 2025A Survey of Context Engineering for Large Language Modelscited by 120
  8. 2024GUI Agents: A Surveycited by 101
  9. 2025A Survey of WebAgents: Towards Next-Generation AI Agents for Web Automation with Large Foundation Modelscited by 87
  10. 2025Self-Challenging Language Model Agentscited by 56
  11. 2025AutoAgent: A Fully-Automated and Zero-Code Framework for LLM Agentscited by 49
  12. 2025REAL: Benchmarking Autonomous Agents on Deterministic Simulations of Real Websitescited by 43
38 more
  1. 2025Infrastructure for AI Agentscited by 40
  2. 2025UFO2: The Desktop AgentOScited by 38
  3. 2025OS Agents: A Survey on MLLM-based Agents for General Computing Devices Usecited by 35
  4. 2025Thinking vs. Doing: Agents that Reason by Scaling Test-Time Interactioncited by 34
  5. 2025CoAct-1: Computer-using Multi-Agent System with Coding Actionscited by 33
  6. 2025Go-Browse: Training Web Agents with Structured Explorationcited by 32
  7. 2025WebChoreArena: Evaluating Web Browsing Agents on Realistic Tedious Web Taskscited by 29
  8. 2025API Agents vs. GUI Agents: Divergence and Convergencecited by 28
  9. 2025Symbiotic Cooperation for Web Agents: Harnessing Complementary Strengths of Large and Small LLMscited by 24
  10. 2026ExpSeek: Self-Triggered Experience Seeking for Web Agentscited by 23
  11. 2024AdaptAgent: Adapting Multimodal Web Agents with Few-Shot Learning from Human Demonstrationscited by 19
  12. 2025Agent-X: Evaluating Deep Multimodal Reasoning in Vision-Centric Agentic Taskscited by 17
  13. 2025Build the web for agents, not agents for the webcited by 17
  14. 2025How Do AI Agents Do Human Work? Comparing AI and Human Workflows Across Diverse Occupationscited by 17
  15. 2026MolmoWeb: Open Visual Web Agent and Open Data for the Open Webcited by 17
  16. 2025ScaleTrack: Scaling and back-tracking Automated GUI Agentscited by 16
  17. 2025WALT: Web Agents that Learn Toolscited by 15
  18. 2025Breaking the Data Barrier -- Building GUI Agents Through Task Generalizationcited by 13
  19. 2025Evolutionary Perspectives on the Evaluation of LLM-Based AI Agents: A Comprehensive Surveycited by 12
  20. 2025Prune4Web: DOM Tree Pruning Programming for Web Agentcited by 11
  21. 2026UI-Voyager: A Self-Evolving GUI Agent Learning via Failed Experiencecited by 11
  22. 2025Evolving in Tasks: Empowering the Multi-modality Large Language Model as the Computer Use Agentcited by 10
  23. 2026OpAgent: Operator Agent for Web Navigationcited by 8
  24. 2025ProSoftArena: Benchmarking Hierarchical Capabilities of Multimodal Agents in Professional Software Environmentscited by 7
  25. 2025Malice in Agentland: Down the Rabbit Hole of Backdoors in the AI Supply Chaincited by 6
  26. 2025Branch-and-Browse: Efficient and Controllable Web Exploration with Tree-Structured Reasoning and Action Memorycited by 6
  27. 2025Experience-Evolving Multi-Turn Tool-Use Agent with Hybrid Episodic-Procedural Memorycited by 6
  28. 2026AgenticShop: Benchmarking Agentic Product Curation for Personalized Web Shoppingcited by 6
  29. 2026Generalization in Online Reinforcement Learning for Mobile Agentscited by 6
  30. 2026Are Online Skill and Memory Modules Always Worth Their Tokens? A Budget-Constrained Study of Web Agentscited by 6
  31. 2025JEF-Hinter: Leveraging Offline Knowledge for Improving Web Agents Adaptationcited by 5
  32. 2025WebDART: Dynamic Decomposition and Re-planning for Complex Web Taskscited by 5
  33. 2025LeechHijack: Covert Computational Resource Exploitation in Intelligent Agent Systemscited by 5
  34. 2026Terminal Agents Suffice for Enterprise Automationcited by 5
  35. 2026On Training Large Language Models for Long-Horizon Tasks: An Empirical Study of Horizon Lengthcited by 5
  36. 2025Doc2Agent: Scalable Generation of Tool-Using Agents from API Documentationcited by 4
  37. 2025MCP vs RAG vs NLWeb vs HTML: A Comparison of the Effectiveness and Efficiency of Different Agent Interfaces to the Web (Technical Report)cited by 4
  38. 2025TheMCPCompany: Creating General-purpose Agents with Task-specific Toolscited by 3

50 most cited of 77 All 77 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2025WebGames: Challenging General-Purpose Web-Browsing AI Agents
  2. 2026Token Efficient Task Execution via Application Behavior Modeling for Web Agents
  3. 2025BrowserAgent: Building Web Agents with Human-Inspired Web Browsing Actions
  4. 2025The Adoption and Usage of AI Agents: Early Evidence from Perplexity
  5. 2025BEARCUBS: A benchmark for computer-using web agents
  6. 2026AutoTailor: Automatic, User-Aligned Capability Selection and Adaptation for Web Agents
  7. 2026Odysseys: Benchmarking Web Agents on Realistic Long Horizon Tasks
  8. 2024WebSuite: Systematically Evaluating Why Web Agents Fail