mlsanityData to 14 Oct 2026

WebShop: Towards Scalable Real-World Web Interaction with Grounded Language Agents

arXiv 2207.01206 ↗First posted 4 Jul 2022cs.CL cs.AI cs.LGcited by 1,251

On the map: LLM agents and tool use → Web, search, and research agents

Abstract (arXiv)

Existing benchmarks for grounding language in interactive environments either lack real-world linguistic elements, or prove difficult to scale up due to substantial human involvement in the collection of data or feedback signals. To bridge this gap, we develop WebShop -- a simulated e-commerce website environment with 1.18 million real-world products and 12,087 crowd-sourced text instructions. Given a text instruction specifying a product requirement, an agent needs to navigate multiple types of webpages and issue diverse actions to find, customize, and purchase an item. WebShop provides several challenges for language grounding including understanding compositional instructions, query (re-)formulation, comprehending and acting on noisy text in webpages, and performing strategic exploration. We collect over 1,600 human demonstrations for the task, and train and evaluate a diverse range of agents using reinforcement learning, imitation learning, and pre-trained image and language models. Our best model achieves a task success rate of 29%, which outperforms rule-based heuristics (9.6%) but is far lower than human expert performance (59%). We also analyze agent and human trajectories and ablate various model components to provide insights for developing future agents with stronger language understanding and decision making abilities. Finally, we show that agents trained on WebShop exhibit non-trivial sim-to-real transfer when evaluated on amazon.com and ebay.com, indicating the potential value of WebShop in developing practical web-based agents that can operate in the wild.

Key facts. Cited by 1,251 papers in the map (data through 14 Oct 2026). Placed in Web, search, and research agents, one of 16 directions in LLM agents and tool use. Builds on 47 of its 65 references that are in the map. First posted on arXiv 4 Jul 2022.

Builds on

47 of 65 references.

References · 47

  1. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  3. 2020Language Models are Few-Shot Learnerscited by 32,419
  4. 2019BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehensioncited by 5,893
  5. 2014Empirical Evaluation of Gated Recurrent Neural Networks on Sequence Modelingcited by 2,367
  6. 2016Asynchronous Methods for Deep Reinforcement Learningcited by 2,258
  7. 2021WebGPT: Browser-assisted question-answering with human feedbackcited by 1,745
  8. 2016OpenAI Gymcited by 1,736
  9. 2020Oscar: Object-Semantics Aligned Pre-training for Vision-Language Taskscited by 1,214
  10. 2017Curiosity-driven Exploration by Self-supervised Predictioncited by 1,194
  11. 2019ALFRED: A Benchmark for Interpreting Grounded Instructions for Everyday Taskscited by 783
  12. 2018MultiWOZ -- A Large-Scale Multi-Domain Wizard-of-Oz Dataset for Task-Oriented Dialogue Modellingcited by 766
35 more
  1. 2021SimVLM: Simple Visual Language Model Pretraining with Weak Supervisioncited by 647
  2. 2018Reinforcement Learning on Web Interfaces Using Workflow-Guided Explorationcited by 284
  3. 2016Bidirectional Attention Flow for Machine Comprehensioncited by 278
  4. 2021Internet-Augmented Dialogue Generationcited by 217
  5. 2019Go-Explore: a New Approach for Hard-Exploration Problemscited by 212
  6. 2019Interactive Fiction Games: A Colossal Adventurecited by 168
  7. 2019A Survey of Reinforcement Learning Informed by Natural Languagecited by 154
  8. 2022Internet-augmented language models through few-shot prompting for open-domain question answeringcited by 131
  9. 2020Task-Oriented Dialogue as Dataflow Synthesiscited by 122
  10. 2020Keep CALM and Explore: Language Models for Action Generation in Text-based Gamescited by 108
  11. 2022Language Models that Seek for Knowledge: Modular Search & Generation for Dialogue and Prompt Completioncited by 102
  12. 2021AndroidEnv: A Reinforcement Learning Platform for Androidcited by 90
  13. 2022A data-driven approach for learning to control computerscited by 88
  14. 2020Multi-agent Communication meets Natural Language: Synergies between Functional and Structural Language Learningcited by 66
  15. 2018Learning to Navigate the Webcited by 60
  16. 2021Pyserini: An Easy-to-Use Python Toolkit to Support Replicable IR Research with Sparse and Dense Representationscited by 60
  17. 2021HTLM: Hyper-Text Pre-Training and Prompting of Language Modelscited by 58
  18. 2018Unsupervised Predictive Memory in a Goal-Directed Agentcited by 55
  19. 2022Bridging the Gap Between Indexing and Retrieval for Differentiable Search Index with Query Generationcited by 48
  20. 2021Towards mental time travel: a hierarchical memory for reinforcement learning agentscited by 40
  21. 2017Task-Oriented Query Reformulation with Reinforcement Learningcited by 39
  22. 2019Generalization of Reinforcement Learners with Working and Episodic Memorycited by 36
  23. 2021Survey on reinforcement learning for language processingcited by 31
  24. 2020Interactive Fiction Game Playing as Multi-Paragraph Reading Comprehension with Reinforcement Learningcited by 27
  25. 2019DOM-Q-NET: Grounded RL on Structured Languagecited by 25
  26. 2018Mapping Natural Language Commands to Web Elementscited by 23
  27. 2022Multi-Stage Episodic Control for Strategic Exploration in Text Gamescited by 22
  28. 2021Reading and Acting while Blindfolded: The Need for Semantics in Text Game Agentscited by 20
  29. 2021Boosting Search Engines with Interactive Agentscited by 19
  30. 2020FLIN: A Flexible Natural Language Interface for Web Navigationcited by 18
  31. 2016Improving Information Extraction by Acquiring External Evidence with Reinforcement Learningcited by 16
  32. 2019Interactive Machine Comprehension with Information Seeking Agentscited by 13
  33. 2021Adversarial Environment Generation for Learning to Navigate the Webcited by 13
  34. 2016End-to-End Goal-Driven Web Navigationcited by 10
  35. 2020Deep Reinforced Query Reformulation for Information Retrievalcited by 10

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2022: 2; 2023: 75; 2024: 214; 2025: 319; 2026: 641.

1,251 papers.

Sort:

By area:

  1. 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,695
  2. 2023Reflexion: Language Agents with Verbal Reinforcement Learningcited by 4,276
  3. 2023SWE-bench: Can Language Models Resolve Real-World GitHub Issues?cited by 3,373
  4. 2023A Survey on Large Language Model based Autonomous Agentscited by 2,242
  5. 2023WebArena: A Realistic Web Environment for Building Autonomous Agentscited by 2,006
  6. 2023The Rise and Potential of Large Language Model Based Agents: A Surveycited by 1,439
  7. 2024τ-bench: A Benchmark for Tool-Agent-User Interaction in Real-World Domainscited by 1,402
  8. 2023AgentBench: Evaluating LLMs as Agentscited by 1,267
  9. 2023Mind2Web: Towards a Generalist Agent for the Webcited by 1,242
  10. 2024ChatGLM: A Family of Large Language Models from GLM-130B to GLM-4 All Toolscited by 1,121
  11. 2023ExpeL: LLM Agents Are Experiential Learnerscited by 876
  12. 2023CogAgent: A Visual Language Model for GUI Agentscited by 704
38 more
  1. 2025BrowseComp: A Simple Yet Challenging Benchmark for Browsing Agentscited by 647
  2. 2025τ2-Bench: Evaluating Conversational Agents in a Dual-Control Environmentcited by 627
  3. 2023Language Agent Tree Search Unifies Reasoning Acting and Planning in Language Modelscited by 574
  4. 2024Executable Code Actions Elicit Better LLM Agentscited by 564
  5. 2024GPT-4V(ision) is a Generalist Web Agent, if Groundedcited by 563
  6. 2023Identifying the Risks of LM Agents with an LM-Emulated Sandboxcited by 548
  7. 2025Group-in-Group Policy Optimization for LLM Agent Trainingcited by 497
  8. 2023Cognitive Architectures for Language Agentscited by 429
  9. 2023Augmented Language Models: a Surveycited by 403
  10. 2025RAGEN: Understanding Self-Evolution in LLM Agents via Multi-Turn Reinforcement Learningcited by 386
  11. 2024Agent Workflow Memorycited by 383
  12. 2024Navigating the Digital World as Humans Do: Universal Visual Grounding for GUI Agentscited by 382
  13. 2023Language Models can Solve Computer Taskscited by 372
  14. 2024WorkArena: How Capable Are Web Agents at Solving Common Knowledge Work Tasks?cited by 363
  15. 2023A Real-World WebAgent with Planning, Long Context Understanding, and Program Synthesiscited by 353
  16. 2024AgentBoard: An Analytical Evaluation Board of Multi-turn LLM Agentscited by 339
  17. 2023AgentTuning: Enabling Generalized Agent Abilities for LLMscited by 338
  18. 2024VideoAgent: Long-form Video Understanding with Large Language Model as Agentcited by 332
  19. 2023Large Language Models for Information Retrieval: A Surveycited by 331
  20. 2025From System 1 to System 2: A Survey of Reasoning Large Language Modelscited by 320
  21. 2025Memory in the Age of AI Agentscited by 303
  22. 2023Tool Learning with Foundation Modelscited by 302
  23. 2024ToolSandbox: A Stateful, Conversational, Interactive Evaluation Benchmark for LLM Tool Use Capabilitiescited by 295
  24. 2024R-Judge: Benchmarking Safety Risk Awareness for LLM Agentscited by 288
  25. 2025SWE-smith: Scaling Data for Software Engineering Agentscited by 285
  26. 2026SkillRL: Evolving Agents via Recursive Skill-Augmented Reinforcement Learningcited by 278
  27. 2024Personal LLM Agents: Insights and Survey about the Capability, Efficiency and Securitycited by 260
  28. 2023FireAct: Toward Language Agent Fine-tuningcited by 256
  29. 2024WebRL: Training LLM Web Agents via Self-Evolving Online Curriculum Reinforcement Learningcited by 256
  30. 2023Auto-GPT for Online Decision Making: Benchmarks and Additional Opinionscited by 255
  31. 2025ScreenSpot-Pro: GUI Grounding for Professional High-Resolution Computer Usecited by 254
  32. 2023InterCode: Standardizing and Benchmarking Interactive Coding with Execution Feedbackcited by 238
  33. 2025MEM1: Learning to Synergize Memory and Reasoning for Efficient Long-Horizon Agentscited by 238
  34. 2024ArCHer: Training Language Model Agents via Hierarchical Multi-Turn RLcited by 228
  35. 2024Tool Learning with Large Language Models: A Surveycited by 223
  36. 2025A Survey on Test-Time Scaling in Large Language Models: What, How, Where, and How Well?cited by 223
  37. 2024AI Agents That Mattercited by 219
  38. 2024Two Tales of Persona in LLMs: A Survey of Role-Playing and Personalizationcited by 208

50 most cited of 1,251 All 1,251 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2026RealWorldShop: Benchmarking and Improving Conversational Shopping Agents in Real-World E-commerce
  2. 2024NNetNav: Unsupervised Learning of Browser Agents Through Environment Interaction in the Wild
  3. 2026WebGym: Scaling Training Environments for Visual Web Agents with Realistic Tasks
  4. 2025Grounding Computer Use Agents on Human Demonstrations
  5. 2026InfiniteWeb: Scalable Web Environment Synthesis for GUI Agent Training
  6. 2026WebWorld: A Large-Scale World Model for Web Agent Training
  7. 2024Large Language Models Can Self-Improve At Web Agent Tasks
  8. 2024AdaptAgent: Adapting Multimodal Web Agents with Few-Shot Learning from Human Demonstrations