mlsanityData to 14 Oct 2026

Mind2Web: Towards a Generalist Agent for the Web

arXiv 2306.06070 ↗First posted 9 Jun 2023cs.CLcited by 1,242

On the map: LLM agents and tool use → Web, search, and research agents

Abstract (arXiv)

We introduce Mind2Web, the first dataset for developing and evaluating generalist agents for the web that can follow language instructions to complete complex tasks on any website. Existing datasets for web agents either use simulated websites or only cover a limited set of websites and tasks, thus not suitable for generalist web agents. With over 2,000 open-ended tasks collected from 137 websites spanning 31 domains and crowdsourced action sequences for the tasks, Mind2Web provides three necessary ingredients for building generalist web agents: 1) diverse domains, websites, and tasks, 2) use of real-world websites instead of simulated and simplified ones, and 3) a broad spectrum of user interaction patterns. Based on Mind2Web, we conduct an initial exploration of using large language models (LLMs) for building generalist web agents. While the raw HTML of real-world websites are often too large to be fed to LLMs, we show that first filtering it with a small LM significantly improves the effectiveness and efficiency of LLMs. Our solution demonstrates a decent level of performance, even on websites or entire domains the model has never seen before, but there is still a substantial room to improve towards truly generalizable agents. We open-source our dataset, model implementation, and trained models (https://osu-nlp-group.github.io/Mind2Web) to facilitate further research on building a generalist agent for the web.

Key facts. Cited by 1,242 papers in the map (data through 14 Oct 2026). Placed in Web, search, and research agents, one of 16 directions in LLM agents and tool use. Builds on 30 of its 43 references that are in the map. First posted on arXiv 9 Jun 2023.

Builds on

30 of 43 references.

Closest ancestors · 1

Same direction, newest first

  1. 2022WebShop: Towards Scalable Real-World Web Interaction with Grounded Language Agents4 Jul

Other references · 29

  1. 2020Language Models are Few-Shot Learnerscited by 32,419
  2. 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,170
  3. 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,695
  4. 2023Toolformer: Language Models Can Teach Themselves to Use Toolscited by 4,182
  5. 2021On the Opportunities and Risks of Foundation Modelscited by 4,033
  6. 2020Dense Passage Retrieval for Open-Domain Question Answeringcited by 3,824
  7. 2022Scaling Instruction-Finetuned Language Modelscited by 3,024
  8. 2022Do As I Can, Not As I Say: Grounding Language in Robotic Affordancescited by 2,348
  9. 2020DeBERTa: Decoding-enhanced BERT with Disentangled Attentioncited by 2,181
  10. 2023Gorilla: Large Language Model Connected with Massive APIscited by 1,332
  11. 2023HuggingGPT: Solving AI Tasks with ChatGPT and its Friends in Hugging Facecited by 1,190
  12. 2018Spider: A Large-Scale Human-Labeled Dataset for Complex and Cross-Domain Semantic Parsing and Text-to-SQL Taskcited by 1,139
17 more
  1. 2019ALFRED: A Benchmark for Interpreting Grounded Instructions for Everyday Taskscited by 783
  2. 2017Reading Wikipedia to Answer Open-Domain Questionscited by 777
  3. 2022LLM-Planner: Few-Shot Grounded Planning for Embodied Agents with Large Language Modelscited by 462
  4. 2019RAT-SQL: Relation-Aware Schema Encoding and Linking for Text-to-SQL Parserscited by 424
  5. 2023Augmented Language Models: a Surveycited by 403
  6. 2023A Comprehensive Survey on Pretrained Foundation Models: A History from BERT to ChatGPTcited by 368
  7. 2023StructGPT: A General Framework for Large Language Model to Reason over Structured Datacited by 343
  8. 2023Tool Learning with Foundation Modelscited by 302
  9. 2018Reinforcement Learning on Web Interfaces Using Workflow-Guided Explorationcited by 284
  10. 2023ToolkenGPT: Augmenting Frozen Language Models with Massive Tools via Tool Embeddingscited by 240
  11. 2023OpenAGI: When LLM Meets Domain Expertscited by 197
  12. 2020Mapping Natural Language Instructions to Mobile UI Action Sequencescited by 181
  13. 2020Beyond I.I.D.: Three Levels of Generalization for Question Answering on Knowledge Basescited by 171
  14. 2022META-GUI: Towards Multi-modal Conversational Agents on Mobile GUIcited by 81
  15. 2022A Dataset for Interactive Vision-Language Navigation with Unknown Command Feasibilitycited by 61
  16. 2022Don't Generate, Discriminate: A Proposal for Grounding Language Models to Real-World Environmentscited by 57
  17. 2021Grounding Open-Domain Instructions to Automate Web Support Taskscited by 43

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 46; 2024: 202; 2025: 382; 2026: 612.

1,242 papers.

Sort:

By area:

  1. 2023SWE-bench: Can Language Models Resolve Real-World GitHub Issues?cited by 3,373
  2. 2023A Survey on Large Language Model based Autonomous Agentscited by 2,242
  3. 2023WebArena: A Realistic Web Environment for Building Autonomous Agentscited by 2,006
  4. 2023The Rise and Potential of Large Language Model Based Agents: A Surveycited by 1,439
  5. 2023AgentBench: Evaluating LLMs as Agentscited by 1,267
  6. 2024OSWorld: Benchmarking Multimodal Agents for Open-Ended Tasks in Real Computer Environmentscited by 1,195
  7. 2024ChatGLM: A Family of Large Language Models from GLM-130B to GLM-4 All Toolscited by 1,121
  8. 2025A-MEM: Agentic Memory for LLM Agentscited by 1,063
  9. 2023CogAgent: A Visual Language Model for GUI Agentscited by 704
  10. 2023Language Agent Tree Search Unifies Reasoning Acting and Planning in Language Modelscited by 574
  11. 2024SeeClick: Harnessing GUI Grounding for Advanced Visual GUI Agentscited by 567
  12. 2024GPT-4V(ision) is a Generalist Web Agent, if Groundedcited by 563
38 more
  1. 2024InjecAgent: Benchmarking Indirect Prompt Injections in Tool-Integrated Large Language Model Agentscited by 538
  2. 2024WebVoyager: Building an End-to-End Web Agent with Large Multimodal Modelscited by 438
  3. 2023Cognitive Architectures for Language Agentscited by 429
  4. 2024Understanding the planning of LLM agents: A surveycited by 421
  5. 2023AppAgent: Multimodal Agents as Smartphone Userscited by 415
  6. 2024TravelPlanner: A Benchmark for Real-World Planning with Language Agentscited by 392
  7. 2024Agent Workflow Memorycited by 383
  8. 2024Navigating the Digital World as Humans Do: Universal Visual Grounding for GUI Agentscited by 382
  9. 2024OS-ATLAS: A Foundation Action Model for Generalist GUI Agentscited by 378
  10. 2024WorkArena: How Capable Are Web Agents at Solving Common Knowledge Work Tasks?cited by 363
  11. 2023A Real-World WebAgent with Planning, Long Context Understanding, and Program Synthesiscited by 353
  12. 2024AgentBoard: An Analytical Evaluation Board of Multi-turn LLM Agentscited by 339
  13. 2023AgentTuning: Enabling Generalized Agent Abilities for LLMscited by 338
  14. 2023Large Language Models for Information Retrieval: A Surveycited by 331
  15. 2024TheAgentCompany: Benchmarking LLM Agents on Consequential Real World Taskscited by 326
  16. 2024Agent-SafetyBench: Evaluating the Safety of LLM Agentscited by 306
  17. 2023MINT: Evaluating LLMs in Multi-turn Interaction with Tools and Language Feedbackcited by 297
  18. 2024ToolSandbox: A Stateful, Conversational, Interactive Evaluation Benchmark for LLM Tool Use Capabilitiescited by 295
  19. 2025ReasoningBank: Scaling Agent Self-Evolving with Reasoning Memorycited by 284
  20. 2024Aguvis: Unified Pure Vision Agents for Autonomous GUI Interactioncited by 278
  21. 2024Personal LLM Agents: Insights and Survey about the Capability, Efficiency and Securitycited by 260
  22. 2023Large Language Models Empowered Agent-based Modeling and Simulation: A Survey and Perspectivescited by 257
  23. 2023FireAct: Toward Language Agent Fine-tuningcited by 256
  24. 2024WebRL: Training LLM Web Agents via Self-Evolving Online Curriculum Reinforcement Learningcited by 256
  25. 2025ScreenSpot-Pro: GUI Grounding for Professional High-Resolution Computer Usecited by 254
  26. 2025MEM1: Learning to Synergize Memory and Reasoning for Efficient Long-Horizon Agentscited by 238
  27. 2024ShowUI: One Vision-Language-Action Model for GUI Visual Agentcited by 232
  28. 2024Windows Agent Arena: Evaluating Multi-Modal OS Agents at Scalecited by 223
  29. 2024AI Agents That Mattercited by 219
  30. 2023MetaTool Benchmark for Large Language Models: Deciding Whether to Use Tools and Which to Usecited by 217
  31. 2025UI-TARS-2 Technical Report: Advancing GUI Agent with Multi-Turn Reinforcement Learningcited by 215
  32. 2024Two Tales of Persona in LLMs: A Survey of Role-Playing and Personalizationcited by 208
  33. 2024Agent Q: Advanced Reasoning and Learning for Autonomous AI Agentscited by 208
  34. 2024OS-Copilot: Towards Generalist Computer Agents with Self-Improvementcited by 205
  35. 2025Large Language Model Agent: A Survey on Methodology, Applications and Challengescited by 205
  36. 2025Survey on Evaluation of LLM-based Agentscited by 204
  37. 2024APIGen: Automated Pipeline for Generating Verifiable and Diverse Function-Calling Datasetscited by 198
  38. 2024OmniParser for Pure Vision Based GUI Agentcited by 198

50 most cited of 1,242 All 1,242 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2026Improving LLM-based Autonomous Web Agents with Filtering
  2. 2025Mind2Web 2: Evaluating Agentic Search with Agent-as-a-Judge
  3. 2024ScribeAgent: Towards Specialized Web Agents Using Production-Scale Workflow Data
  4. 2025BrowserAgent: Building Web Agents with Human-Inspired Web Browsing Actions
  5. 2026Weasel: Out-of-Domain Generalization for Web Agents via Importance-Diversity Data Selection
  6. 2024WILBUR: Adaptive In-Context Learning for Robust and Accurate Web Agents
  7. 2024Large Language Models Can Self-Improve At Web Agent Tasks
  8. 2026Persona2Web: Benchmarking Personalized Web Agents for Contextual Reasoning with User History