mlsanityData to 7 Oct 2026

Is Your LLM Secretly a World Model of the Internet? Model-Based Planning for Web Agents

arXiv 2411.06559 ↗First posted 10 Nov 2024cs.AIcited by 144

On the map: World models → World models for language agents

also in 1 more
Abstract (arXiv)

Language agents based on large language models (LLMs) have demonstrated great promise in automating web-based tasks. Recent work has shown that incorporating advanced planning algorithms, e.g., tree search, is advantageous over reactive planning for web agents. However, unlike simulated sandbox environments, real-world environments such as the web are rife with irreversible actions. This undermines the feasibility of backtracking, a cornerstone of (tree) search. Overly relying on test-time search also hurts efficiency. We advocate model-based planning for web agents that employs a world model to simulate and deliberate over the outcome of each candidate action before committing to one. We systematically explore this paradigm by (1) Proposing a model-based planning framework, WebDreamer, which employs LLMs to serve as both world models and value functions; (2) Training specialized LLMs as world models with a scalable data synthesis pipeline. Empirical results demonstrate that WebDreamer achieves substantial performance improvements over reactive baselines. It is competitive, while being 4-5 times more efficient, with tree search in sandbox environments (VisualWebArena) and also works effectively on real-world websites (Online-Mind2Web and Mind2Web-Live). Furthermore, our trained world model, Dreamer-7B, performs comparable to GPT-4o, highlighting the potential of specialized world models for efficient and effective planning in complex web environments.

Key facts. Cited by 144 papers in the map (data through 7 Oct 2026). Placed in World models for language agents, one of 11 directions in World models. Builds on 39 of its 54 references that are in the map. First posted on arXiv 10 Nov 2024.

Builds on

39 of 54 references.

Closest ancestors · 2

Same direction, newest first

  1. 2024Web Agents with World Models: Learning and Leveraging Environment Dynamics in Web Navigation17 Oct
  2. 2023Reasoning with Language Model is Planning with World Model24 May

Other references · 37

  1. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,302
  2. 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,355
  3. 2024Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolutioncited by 4,151
  4. 2023Tree of Thoughts: Deliberate Problem Solving with Large Language Modelscited by 3,155
  5. 2023WebArena: A Realistic Web Environment for Building Autonomous Agentscited by 1,908
  6. 2019Dream to Control: Learning Behaviors by Latent Imaginationcited by 1,598
  7. 2019Mastering Atari, Go, Chess and Shogi by Planning with a Learned Modelcited by 1,327
  8. 2023Mind2Web: Towards a Generalist Agent for the Webcited by 1,210
  9. 2022WebShop: Towards Scalable Real-World Web Interaction with Grounded Language Agentscited by 1,197
  10. 2020Mastering Atari with Discrete World Modelscited by 990
  11. 2024Large Language Monkeys: Scaling Inference Compute with Repeated Samplingcited by 946
  12. 2018Recurrent World Models Facilitate Policy Evolutioncited by 761
25 more
  1. 2023CogAgent: A Visual Language Model for GUI Agentscited by 692
  2. 2024SeeClick: Harnessing GUI Grounding for Advanced Visual GUI Agentscited by 553
  3. 2024GPT-4V(ision) is a Generalist Web Agent, if Groundedcited by 547
  4. 2024WebVoyager: Building an End-to-End Web Agent with Large Multimodal Modelscited by 424
  5. 2024Navigating the Digital World as Humans Do: Universal Visual Grounding for GUI Agentscited by 373
  6. 2022Pix2Struct: Screenshot Parsing as Pretraining for Visual Language Understandingcited by 360
  7. 2023A Real-World WebAgent with Planning, Long Context Understanding, and Program Synthesiscited by 350
  8. 2023Alphazero-like Tree-Search can Guide Large Language Model Decoding and Trainingcited by 291
  9. 2023Set-of-Mark Prompting Unleashes Extraordinary Visual Grounding in GPT-4Vcited by 291
  10. 2018Reinforcement Learning on Web Interfaces Using Workflow-Guided Explorationcited by 282
  11. 2024Agent Q: Advanced Reasoning and Learning for Autonomous AI Agentscited by 201
  12. 2024Trial and Error: Exploration-Based Trajectory Optimization for LLM Agentscited by 194
  13. 2024Ferret-UI: Grounded Mobile UI Understanding with Multimodal LLMscited by 173
  14. 2023Multimodal Web Navigation with Instruction-Finetuned Foundation Modelscited by 159
  15. 2024Tree Search for Language Model Agentscited by 148
  16. 2024Autonomous Evaluation and Refinement of Digital Agentscited by 136
  17. 2024AutoWebGLM: A Large Language Model-based Web Navigating Agentcited by 130
  18. 2024WebCanvas: Benchmarking Web Agents in Online Environmentscited by 119
  19. 2024WebPilot: A Versatile and Autonomous Multi-Agent System for Web Task Execution with Strategic Explorationcited by 100
  20. 2024ScreenAI: A Vision-Language Model for UI and Infographics Understandingcited by 97
  21. 2024Planning In Natural Language Improves LLM Search For Code Generationcited by 79
  22. 2023From Pixels to UI Actions: Learning to Follow Instructions via Graphical User Interfacescited by 67
  23. 2022Don't Generate, Discriminate: A Proposal for Grounding Language Models to Real-World Environmentscited by 57
  24. 2024Large Language Models Can Self-Improve At Web Agent Taskscited by 54
  25. 2017Learning model-based planning from scratchcited by 21

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 5; 2025: 65; 2026: 74.

144 papers.

Sort:

By area:

  1. 2025Plan-and-Act: Improving Planning of Agents for Long-Horizon Taskscited by 190
  2. 2025A Survey of Reinforcement Learning for Large Reasoning Modelscited by 175
  3. 2025SkillWeaver: Web Agents can Self-Improve by Discovering and Honing Skillscited by 174
  4. 2025An Illusion of Progress? Assessing the Current State of Web Agentscited by 165
  5. 2024Understanding World or Predicting Future? A Comprehensive Survey of World Modelscited by 161
  6. 2024Large Language Model-Brained GUI Agents: A Surveycited by 160
  7. 2024The BrowserGym Ecosystem for Web Agent Researchcited by 152
  8. 2025Advances and Challenges in Foundation Agents: From Brain-Inspired Intelligence to Evolutionary, Collaborative, and Safe Systemscited by 125
  9. 2024GUI Agents: A Surveycited by 101
  10. 2025A Survey of WebAgents: Towards Next-Generation AI Agents for Web Automation with Large Foundation Modelscited by 87
  11. 2025AGrail: A Lifelong Agent Guardrail with Effective and Adaptive Safety Detectioncited by 82
  12. 2025Agent Learning via Early Experiencecited by 79
38 more
  1. 2025Deep Research: A Survey of Autonomous Research Agentscited by 63
  2. 2025WebEvolver: Enhancing Web Agent Self-Improvement with Coevolving World Modelcited by 56
  3. 2025SeRL: Self-Play Reinforcement Learning for Large Language Models with Limited Datacited by 50
  4. 2026Expanding the Capabilities of Reinforcement Learning via Text Feedbackcited by 49
  5. 2025REAL: Benchmarking Autonomous Agents on Deterministic Simulations of Real Websitescited by 43
  6. 2025WMNav: Integrating Vision-Language Models into World Models for Object Goal Navigationcited by 42
  7. 2025Web-Shepherd: Advancing PRMs for Reinforcing Web Agentscited by 41
  8. 2025Scaling Agent Learning via Experience Synthesiscited by 40
  9. 2025From Word to World: Can Large Language Models be Implicit Text-based World Models?cited by 40
  10. 2025RLVR-World: Training World Models with Reinforcement Learningcited by 38
  11. 2025Large Language Models for Planning: A Comprehensive and Systematic Surveycited by 38
  12. 2025Planning with Reasoning using Vision Language World Modelcited by 36
  13. 2025Towards Trustworthy GUI Agents: A Surveycited by 33
  14. 2025ViMo: A Generative Visual GUI World Model for App Agentscited by 33
  15. 2026Code2World: A GUI World Model via Renderable Code Generationcited by 26
  16. 2025PolySkill: Learning Generalizable Skills Through Polymorphic Abstractioncited by 25
  17. 2026TimeART: Towards Agentic Time Series Reasoning via Tool-Augmentationcited by 25
  18. 2026WebWorld: A Large-Scale World Model for Web Agent Trainingcited by 23
  19. 2026Qwen-AgentWorld: Language World Models for General Agentscited by 22
  20. 2026MobileDreamer: Generative Sketch World Model for GUI Agentcited by 21
  21. 2026Reinforcement World Model Learning for LLM-based Agentscited by 21
  22. 2025WebGuard: Building a Generalizable Guardrail for Web Agentscited by 20
  23. 2025Scaling Autonomous Agents via Automatic Reward Modeling And Planningcited by 19
  24. 2025VEM: Environment-Free Exploration for Training GUI Agent with Value Environment Modelcited by 19
  25. 2025Agent-RewardBench: Towards a Unified Benchmark for Reward Modeling across Perception, Planning, and Safety in Real-World Multimodal Agentscited by 19
  26. 2025WebSynthesis: World-Model-Guided MCTS for Efficient WebUI-Trajectory Synthesiscited by 18
  27. 2026ToolTree: Efficient LLM Agent Tool Planning via Dual-Feedback Monte Carlo Tree Search and Bidirectional Pruningcited by 18
  28. 2025Build the web for agents, not agents for the webcited by 17
  29. 2025LLMs as Scalable, General-Purpose Simulators For Evolving Digital Agent Trainingcited by 17
  30. 2026K-Search: LLM Kernel Generation via Co-Evolving Intrinsic World Modelcited by 17
  31. 2025ScaleTrack: Scaling and back-tracking Automated GUI Agentscited by 16
  32. 2025FocusAgent: Simple Yet Effective Ways of Trimming the Large Context of Web Agentscited by 16
  33. 2025WALT: Web Agents that Learn Toolscited by 15
  34. 2025R2D2: Remembering, Replaying and Dynamic Decision Making with a Reflective Agentic Memorycited by 14
  35. 2025WebCoT: Enhancing Web Agent Reasoning by Reconstructing Chain-of-Thought in Reflection, Branching, and Rollbackcited by 14
  36. 2025Dyna-Think: Synergizing Reasoning, Acting, and World Model Simulation in AI Agentscited by 14
  37. 2025General Agentic Planning Through Simulative Reasoning with World Modelscited by 13
  38. 2025SSRL: Self-Search Reinforcement Learningcited by 13

50 most cited of 144 All 144 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2026Why Do LLM-based Web Agents Fail? A Hierarchical Planning Perspective
  2. 2026AI Planning Framework for LLM-Based Web Agents
  3. 2026Improving LLM-based Autonomous Web Agents with Filtering
  4. 2024Large Language Model based Multi-Agents: A Survey of Progress and Challenges
  5. 2025Symbiotic Cooperation for Web Agents: Harnessing Complementary Strengths of Large and Small LLMs
  6. 2024Understanding the planning of LLM agents: A survey
  7. 2023A Survey on Large Language Model based Autonomous Agents
  8. 2024AgentOccam: A Simple Yet Strong Baseline for LLM-Based Web Agents