# A Survey of WebAgents: Towards Next-Generation AI Agents for Web Automation with Large Foundation Models

arXiv: https://arxiv.org/abs/2503.23350 · HTML: https://mlsanity.com/paper/2503.23350/

## Key facts

- Cited by 87 papers in the map (data through 14 Oct 2026).
- Placed in Web, search, and research agents, one of 16 directions in LLM agents and tool use.
- Builds on 105 of its 185 references that are in the map.
- First posted on arXiv 30 Mar 2025.

## Placed in

- LLM agents and tool use / [Web, search, and research agents](https://mlsanity.com/agents/web-search-and-research-agents/)

## Builds on (most cited first, 30 of 105)

- [WebArena: A Realistic Web Environment for Building Autonomous Agents](https://mlsanity.com/paper/2307.13854/) (2023)
- [AutoGen: Enabling Next-Gen LLM Applications via Multi-Agent Conversation](https://mlsanity.com/paper/2308.08155/) (2023)
- [Mind2Web: Towards a Generalist Agent for the Web](https://mlsanity.com/paper/2306.06070/) (2023)
- [Deep Reinforcement Learning for Autonomous Driving: A Survey](https://mlsanity.com/paper/2002.00444/) (2020)
- [Multi-Agent Reinforcement Learning: A Selective Overview of Theories and Algorithms](https://mlsanity.com/paper/1911.10635/) (2019)
- [SeeClick: Harnessing GUI Grounding for Advanced Visual GUI Agents](https://mlsanity.com/paper/2401.10935/) (2024)
- [GPT-4V(ision) is a Generalist Web Agent, if Grounded](https://mlsanity.com/paper/2401.01614/) (2024)
- [MemoryBank: Enhancing Large Language Models with Long-Term Memory](https://mlsanity.com/paper/2305.10250/) (2023)
- [A Survey on RAG Meeting LLMs: Towards Retrieval-Augmented Large Language Models](https://mlsanity.com/paper/2405.06211/) (2024)
- [WebVoyager: Building an End-to-End Web Agent with Large Multimodal Models](https://mlsanity.com/paper/2401.13919/) (2024)
- [Understanding the planning of LLM agents: A survey](https://mlsanity.com/paper/2402.02716/) (2024)
- [Navigating the Digital World as Humans Do: Universal Visual Grounding for GUI Agents](https://mlsanity.com/paper/2410.05243/) (2024)
- [A Survey on Large Language Models for Recommendation](https://mlsanity.com/paper/2305.19860/) (2023)
- [OS-ATLAS: A Foundation Action Model for Generalist GUI Agents](https://mlsanity.com/paper/2410.23218/) (2024)
- [Language Models can Solve Computer Tasks](https://mlsanity.com/paper/2303.17491/) (2023)
- [Pix2Struct: Screenshot Parsing as Pretraining for Visual Language Understanding](https://mlsanity.com/paper/2210.03347/) (2022)
- [A Real-World WebAgent with Planning, Long Context Understanding, and Program Synthesis](https://mlsanity.com/paper/2307.12856/) (2023)
- [Android in the Wild: A Large-Scale Dataset for Android Device Control](https://mlsanity.com/paper/2307.10088/) (2023)
- [Set-of-Mark Prompting Unleashes Extraordinary Visual Grounding in GPT-4V](https://mlsanity.com/paper/2310.11441/) (2023)
- [R-Judge: Benchmarking Safety Risk Awareness for LLM Agents](https://mlsanity.com/paper/2401.10019/) (2024)
- [Personal LLM Agents: Insights and Survey about the Capability, Efficiency and Security](https://mlsanity.com/paper/2401.05459/) (2024)
- [DigiRL: Training In-The-Wild Device-Control Agents with Autonomous Reinforcement Learning](https://mlsanity.com/paper/2406.11896/) (2024)
- [OS-Copilot: Towards Generalist Computer Agents with Self-Improvement](https://mlsanity.com/paper/2402.07456/) (2024)
- [OmniParser for Pure Vision Based GUI Agent](https://mlsanity.com/paper/2408.00203/) (2024)
- [Plan-and-Act: Improving Planning of Agents for Long-Horizon Tasks](https://mlsanity.com/paper/2503.09572/) (2025)
- [On the Effects of Data Scale on UI Control Agents](https://mlsanity.com/paper/2406.03679/) (2024)
- [Synapse: Trajectory-as-Exemplar Prompting with Memory for Computer Control](https://mlsanity.com/paper/2306.07863/) (2023)
- [The BrowserGym Ecosystem for Web Agent Research](https://mlsanity.com/paper/2412.05467/) (2024)
- [Multimodal Web Navigation with Instruction-Finetuned Foundation Models](https://mlsanity.com/paper/2305.11854/) (2023)
- [Android in the Zoo: Chain-of-Action-Thought for GUI Agents](https://mlsanity.com/paper/2403.02713/) (2024)

## Built on by (most cited first, 30 of 87)

- [AI Agents vs. Agentic AI: A Conceptual Taxonomy, Applications and Challenges](https://mlsanity.com/paper/2505.10468/) (2025)
- [Information Gain-based Policy Optimization: A Simple and Effective Approach for Multi-Turn Search Agents](https://mlsanity.com/paper/2510.14967/) (2025)
- [DeepShop: A Benchmark for Deep Research Shopping Agents](https://mlsanity.com/paper/2506.02839/) (2025)
- [GenEnv: Difficulty-Aligned Co-Evolution Between LLM Agents and Environment Simulators](https://mlsanity.com/paper/2512.19682/) (2025)
- [The 2025 AI Agent Index: Documenting Technical and Safety Features of Deployed Agentic AI Systems](https://mlsanity.com/paper/2602.17753/) (2026)
- [Why Do LLM Agents Fail in Exploring New Environments? A World-Modeling Perspective](https://mlsanity.com/paper/2510.15047/) (2025)
- [WebXSkill: Skill Learning for Autonomous Web Agents](https://mlsanity.com/paper/2604.13318/) (2026)
- [ExpSeek: Self-Triggered Experience Seeking for Web Agents](https://mlsanity.com/paper/2601.08605/) (2026)
- [SecureWebArena: A Holistic Security Evaluation Benchmark for LVLM-based Web Agents](https://mlsanity.com/paper/2510.10073/) (2025)
- [WebMall -- A Multi-Shop Benchmark for Evaluating Web Agents](https://mlsanity.com/paper/2508.13024/) (2025)
- [Agent Data Protocol: Unifying Datasets for Diverse, Effective Fine-tuning of LLM Agents](https://mlsanity.com/paper/2510.24702/) (2025)
- [InfoFlow: Reinforcing Search Agent Via Reward Density Optimization](https://mlsanity.com/paper/2510.26575/) (2025)
- [The Why Behind the Action: Unveiling Internal Drivers via Agentic Attribution](https://mlsanity.com/paper/2601.15075/) (2026)
- [WebOperator: Action-Aware Tree Search for Autonomous Agents in Web Environment](https://mlsanity.com/paper/2512.12692/) (2025)
- [SynWorld: Virtual Scenario Synthesis for Agentic Action Knowledge Refinement](https://mlsanity.com/paper/2504.03561/) (2025)
- [Diffusion Generative Recommendation with Continuous Tokens](https://mlsanity.com/paper/2504.12007/) (2025)
- [Agentic Knowledgeable Self-awareness](https://mlsanity.com/paper/2504.03553/) (2025)
- [MiniAppBench: Evaluating the Shift from Text to Interactive HTML Responses in LLM-Powered Assistants](https://mlsanity.com/paper/2603.09652/) (2026)
- [Retrieval-Augmented Purifier for Robust LLM-Empowered Recommendation](https://mlsanity.com/paper/2504.02458/) (2025)
- [InfantAgent-Next: A Multimodal Generalist Agent for Automated Computer Interaction](https://mlsanity.com/paper/2505.10887/) (2025)
- [A Functionality-Grounded Benchmark for Evaluating Web Agents in E-commerce Domains](https://mlsanity.com/paper/2508.15832/) (2025)
- [Exploring Backdoor Attack and Defense for LLM-empowered Recommendations](https://mlsanity.com/paper/2504.11182/) (2025)
- [LaMDAgent: An Autonomous Framework for Post-Training Pipeline Optimization via LLM Agents](https://mlsanity.com/paper/2505.21963/) (2025)
- [Autonomous Data Agents: A New Opportunity for Smart Data](https://mlsanity.com/paper/2509.18710/) (2025)
- [ColorBrowserAgent: Complex Long-Horizon Browser Agent with Adaptive Knowledge Evolution](https://mlsanity.com/paper/2601.07262/) (2026)
- [Atomicity for Agents: Exposing, Exploiting, and Mitigating TOCTOU Vulnerabilities in Browser-Use Agents](https://mlsanity.com/paper/2603.00476/) (2026)
- [Mind the Web: The Security of Web Use Agents](https://mlsanity.com/paper/2506.07153/) (2025)
- [Optimizing Agentic Workflows using Meta-tools](https://mlsanity.com/paper/2601.22037/) (2026)
- [Empowering GUI Agents via Autonomous Experience Exploration and Hindsight Experience Utilization for Task Planning](https://mlsanity.com/paper/2606.27330/) (2026)
- [How do Visual Attributes Influence Web Agents? A Comprehensive Evaluation of User Interface Design Factors](https://mlsanity.com/paper/2601.21961/) (2026)

## Abstract (arXiv)

With the advancement of web techniques, they have significantly revolutionized various aspects of people's lives. Despite the importance of the web, many tasks performed on it are repetitive and time-consuming, negatively impacting overall quality of life. To efficiently handle these tedious daily tasks, one of the most promising approaches is to advance autonomous agents based on Artificial Intelligence (AI) techniques, referred to as AI Agents, as they can operate continuously without fatigue or performance degradation. In the context of the web, leveraging AI Agents -- termed WebAgents -- to automatically assist people in handling tedious daily tasks can dramatically enhance productivity and efficiency. Recently, Large Foundation Models (LFMs) containing billions of parameters have exhibited human-like language understanding and reasoning capabilities, showing proficiency in performing various complex tasks. This naturally raises the question: `Can LFMs be utilized to develop powerful AI Agents that automatically handle web tasks, providing significant convenience to users?' To fully explore the potential of LFMs, extensive research has emerged on WebAgents designed to complete daily web tasks according to user instructions, significantly enhancing the convenience of daily human life. In this survey, we comprehensively review existing research studies on WebAgents across three key aspects: architectures, training, and trustworthiness. Additionally, several promising directions for future research are explored to provide deeper insights.