mlsanityData to 7 Oct 2026

Agent-E: From Autonomous Web Navigation to Foundational Design Principles in Agentic Systems

arXiv 2407.13032 ↗First posted 17 Jul 2024cs.AIcited by 79

On the map: LLM agents and tool use → Web, search, and research agents

Abstract (arXiv)

AI Agents are changing the way work gets done, both in consumer and enterprise domains. However, the design patterns and architectures to build highly capable agents or multi-agent systems are still developing, and the understanding of the implication of various design choices and algorithms is still evolving. In this paper, we present our work on building a novel web agent, Agent-E footnoteOur code is available at urlhttps://github.com/EmergenceAI/Agent-E. Agent-E introduces numerous architectural improvements over prior state-of-the-art web agents such as hierarchical architecture, flexible DOM distillation and denoising method, and the concept of textitchange observation to guide the agent towards more accurate performance. We first present the results of an evaluation of Agent-E on WebVoyager benchmark dataset and show that Agent-E beats other SOTA text and multi-modal web agents on this benchmark in most categories by 10-30%. We then synthesize our learnings from the development of Agent-E into general design principles for developing agentic systems. These include the use of domain-specific primitive skills, the importance of distillation and de-noising of environmental observations, the advantages of a hierarchical architecture, and the role of agentic self-improvement to enhance agent efficiency and efficacy as the agent gathers experience.

Key facts. Cited by 79 papers in the map (data through 7 Oct 2026). Placed in Web, search, and research agents, one of 16 directions in LLM agents and tool use. Builds on 33 of its 40 references that are in the map. First posted on arXiv 17 Jul 2024.

Builds on

33 of 40 references.

Closest ancestors · 6

Same direction, newest first

  1. 2024WILBUR: Adaptive In-Context Learning for Robust and Accurate Web Agents8 Apr
  2. 2024WebVoyager: Building an End-to-End Web Agent with Large Multimodal Models25 Jan
  3. 2024GPT-4V(ision) is a Generalist Web Agent, if Grounded3 Jan
  4. 2023WebArena: A Realistic Web Environment for Building Autonomous Agents25 Jul
  5. 2023Mind2Web: Towards a Generalist Agent for the Web9 Jun
  6. 2022WebShop: Towards Scalable Real-World Web Interaction with Grounded Language Agents4 Jul

Other references · 27

  1. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,302
  2. 2022PaLM: Scaling Language Modeling with Pathwayscited by 5,304
  3. 2023Reflexion: Language Agents with Verbal Reinforcement Learningcited by 4,064
  4. 2022Do As I Can, Not As I Say: Grounding Language in Robotic Affordancescited by 2,302
  5. 2023Voyager: An Open-Ended Embodied Agent with Large Language Modelscited by 2,081
  6. 2023ToolLLM: Facilitating Large Language Models to Master 16000+ Real-world APIscited by 1,880
  7. 2021WebGPT: Browser-assisted question-answering with human feedbackcited by 1,722
  8. 2023Gorilla: Large Language Model Connected with Massive APIscited by 1,302
  9. 2023Large Language Models Cannot Self-Correct Reasoning Yetcited by 965
  10. 2023Textbooks Are All You Need II: phi-1.5 technical reportcited by 559
  11. 2016The Option-Critic Architecturecited by 433
  12. 2023Describe, Explain, Plan and Select: Interactive Planning with Large Language Models Enables Open-World Multi-Task Agentscited by 396
15 more
  1. 2023Unleashing Large-Scale Video Generative Pre-training for Visual Robot Manipulationcited by 353
  2. 2022Video PreTraining (VPT): Learning to Act by Watching Unlabeled Online Videoscited by 338
  3. 2023Specializing Smaller Language Models towards Multi-Step Reasoningcited by 249
  4. 2024DigiRL: Training In-The-Wild Device-Control Agents with Autonomous Reinforcement Learningcited by 210
  5. 2024AI Agents That Mattercited by 199
  6. 2023Navigate through Enigmatic Labyrinth A Survey of Chain of Thought Reasoning: Advances, Frontiers and Futurecited by 184
  7. 2023AutoDroid: LLM-powered Task Automation in Androidcited by 178
  8. 2024LLMs Can't Plan, But Can Help Planning in LLM-Modulo Frameworkscited by 178
  9. 2023JARVIS-1: Open-World Multi-task Agents with Memory-Augmented Multimodal Language Modelscited by 170
  10. 2024Orca-Math: Unlocking the potential of SLMs in Grade School Mathcited by 151
  11. 2023Can Large Language Models Really Improve by Self-critiquing Their Own Plans?cited by 107
  12. 2023Evaluating Cognitive Maps and Planning in Large Language Models with CogEvalcited by 77
  13. 2023Robotic Offline RL from Internet Videos via Value-Function Pre-Trainingcited by 31
  14. 2024Automating the Enterprise with Foundation Modelscited by 14
  15. 2021Compositional Generalization for Natural Language Interfaces to Web APIscited by 7

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 11; 2025: 36; 2026: 32.

79 papers.

Sort:

By area:

  1. 2025UI-TARS: Pioneering Automated GUI Interaction with Native Agentscited by 584
  2. 2025Plan-and-Act: Improving Planning of Agents for Long-Horizon Taskscited by 190
  3. 2025WebWalker: Benchmarking LLMs in Web Traversalcited by 171
  4. 2025An Illusion of Progress? Assessing the Current State of Web Agentscited by 165
  5. 2025WebAgent-R1: Training Web Agents via End-to-End Multi-Turn Reinforcement Learningcited by 162
  6. 2024Large Language Model-Brained GUI Agents: A Surveycited by 160
  7. 2024Aria-UI: Visual Grounding for GUI Instructionscited by 136
  8. 2024ST-WebAgentBench: A Benchmark for Evaluating Safety and Trustworthiness in Web Agentscited by 126
  9. 2024AgentOccam: A Simple Yet Strong Baseline for LLM-Based Web Agentscited by 110
  10. 2025A Comprehensive Survey in LLM(-Agent) Full Stack Safety: Data, Training and Deploymentcited by 110
  11. 2025A Survey of WebAgents: Towards Next-Generation AI Agents for Web Automation with Large Foundation Modelscited by 87
  12. 2025Deep Research: A Survey of Autonomous Research Agentscited by 63
38 more
  1. 2024Ferret-UI 2: Mastering Universal User Interface Understanding Across Platformscited by 51
  2. 2026A Survey of Agentic Reasoning for Large Language Models: Towards Recursively Self-Improving and Collective Agentscited by 47
  3. 2024OpenWebVoyager: Building Multimodal Web Agents via Iterative Real-World Exploration, Feedback and Optimizationcited by 45
  4. 2025DeepShop: A Benchmark for Deep Research Shopping Agentscited by 42
  5. 2025OS Agents: A Survey on MLLM-based Agents for General Computing Devices Usecited by 35
  6. 2025Thinking vs. Doing: Agents that Reason by Scaling Test-Time Interactioncited by 34
  7. 2025Fara-7B: An Efficient Agentic Model for Computer Usecited by 34
  8. 2025AGENTSAFE: Benchmarking the Safety of Embodied Agents on Hazardous Instructionscited by 33
  9. 2026MUZZLE: Adaptive Agentic Red-Teaming of Web Agents Against Indirect Prompt Injection Attackscited by 18
  10. 2025SecureWebArena: A Holistic Security Evaluation Benchmark for LVLM-based Web Agentscited by 17
  11. 2026MolmoWeb: Open Visual Web Agent and Open Data for the Open Webcited by 17
  12. 2025FocusAgent: Simple Yet Effective Ways of Trimming the Large Context of Web Agentscited by 16
  13. 2025LiteWebAgent: The Open-Source Suite for VLM-Based Web-Agent Applicationscited by 14
  14. 2025Evolution of AI in Education: Agentic Workflowscited by 13
  15. 2025Investigating the Impact of Dark Patterns on LLM-Based Web Agentscited by 13
  16. 2025From Code Foundation Models to Agents and Applications: A Comprehensive Survey and Practical Guide to Code Intelligencecited by 13
  17. 2025Evolutionary Perspectives on the Evaluation of LLM-Based AI Agents: A Comprehensive Surveycited by 12
  18. 2026The Evolution of Tool Use in LLM Agents: From Single-Tool Call to Multi-Tool Orchestrationcited by 11
  19. 2024Evaluation-Driven Development and Operations of LLM Agents: A Process Model and Reference Architecturecited by 10
  20. 2025Agent-S: LLM Agentic workflow to automate Standard Operating Procedurescited by 10
  21. 2025GUI-ARP: Enhancing Grounding with Adaptive Region Perception for GUI Agentscited by 10
  22. 2025OpenApps: Simulating Environment Variations to Measure UI-Agent Reliabilitycited by 10
  23. 2026Agentic Environment Engineering for Large Language Models: A Survey of Environment Modeling, Synthesis, Evaluation, and Applicationcited by 8
  24. 2025A Functionality-Grounded Benchmark for Evaluating Web Agents in E-commerce Domainscited by 7
  25. 2026ShowUI-Aloha: Human-Taught GUI Agentcited by 7
  26. 2024DAWN: Designing Distributed Agents in a Worldwide Networkcited by 6
  27. 2025OS-MAP: How Far Can Computer-Using Agents Go in Breadth and Depth?cited by 6
  28. 2026When Bots Take the Bait: Exposing and Mitigating the Emerging Social Engineering Attack in Web Automation Agentcited by 6
  29. 2026AgenticShop: Benchmarking Agentic Product Curation for Personalized Web Shoppingcited by 6
  30. 2026Scaling Web Agent Training through Automatic Data Generation and Fine-grained Evaluationcited by 6
  31. 2026Atomicity for Agents: Exposing, Exploiting, and Mitigating TOCTOU Vulnerabilities in Browser-Use Agentscited by 6
  32. 2024Re-Thinking Process Mining in the AI-Based Agents Eracited by 5
  33. 2024Multimodal Auto Validation For Self-Refinement in Web Agentscited by 5
  34. 2024Accelerating Manufacturing Scale-Up from Material Discovery Using Agentic Web Navigation and Retrieval-Augmented AI for Process Engineering Schematics Designcited by 5
  35. 2025PEAR: Planner-Executor Agent Robustness Benchmarkcited by 5
  36. 2025Surfer 2: The Next Generation of Cross-Platform Computer Use Agentscited by 5
  37. 2026Emergence WebVoyager: Toward Consistent and Transparent Evaluation of (Web) Agents in The Wildcited by 5
  38. 2026Fara-1.5: Scalable Learning Environments for Computer Use Agentscited by 5

50 most cited of 79 All 79 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2025Build the web for agents, not agents for the web
  2. 2025Agentic Web: Weaving the Next Web with AI Agents
  3. 2025REAL: Benchmarking Autonomous Agents on Deterministic Simulations of Real Websites
  4. 2026OpAgent: Operator Agent for Web Navigation
  5. 2025Building the Web for Agents: A Declarative Framework for Agent-Web Interaction
  6. 2025AgentA/B: Automated and Scalable Web A/BTesting with Interactive LLM Agents
  7. 2026Synergy: A Next-Generation General-Purpose Agent for Open Agentic Web
  8. 2026Infrastructure for the Agentic Web: Gap Analysis and Architecture from the Agentverse Platform