mlsanityData to 7 Oct 2026

Web Agents with World Models: Learning and Leveraging Environment Dynamics in Web Navigation

arXiv 2410.13232 ↗First posted 17 Oct 2024cs.CLcited by 116

On the map: World models → World models for language agents

also in 1 more
Abstract (arXiv)

Large language models (LLMs) have recently gained much attention in building autonomous agents. However, the performance of current LLM-based web agents in long-horizon tasks is far from optimal, often yielding errors such as repeatedly buying a non-refundable flight ticket. By contrast, humans can avoid such an irreversible mistake, as we have an awareness of the potential outcomes (e.g., losing money) of our actions, also known as the "world model". Motivated by this, our study first starts with preliminary analyses, confirming the absence of world models in current LLMs (e.g., GPT-4o, Claude-3.5-Sonnet, etc.). Then, we present a World-model-augmented (WMA) web agent, which simulates the outcomes of its actions for better decision-making. To overcome the challenges in training LLMs as world models predicting next observations, such as repeated elements across observations and long HTML inputs, we propose a transition-focused observation abstraction, where the prediction objectives are free-form natural language descriptions exclusively highlighting important state differences between time steps. Experiments on WebArena and Mind2Web show that our world models improve agents' policy selection without training and demonstrate our agents' cost- and time-efficiency compared to recent tree-search-based agents.

Key facts. Cited by 116 papers in the map (data through 7 Oct 2026). Placed in World models for language agents, one of 11 directions in World models. Builds on 38 of its 54 references that are in the map. First posted on arXiv 17 Oct 2024.

Builds on

38 of 54 references.

Closest ancestors · 1

Same direction, newest first

  1. 2024Language-Guided World Models: A Model-Based Approach to AI Control24 Jan

Other references · 37

  1. 2023GPT-4 Technical Reportcited by 18,132
  2. 2024The Llama 3 Herd of Modelscited by 14,192
  3. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,302
  4. 2023Visual Instruction Tuningcited by 7,792
  5. 2023Efficient Memory Management for Large Language Model Serving with PagedAttentioncited by 5,767
  6. 2022Self-Consistency Improves Chain of Thought Reasoning in Language Modelscited by 5,601
  7. 2023Reflexion: Language Agents with Verbal Reinforcement Learningcited by 4,064
  8. 2023Self-Refine: Iterative Refinement with Self-Feedbackcited by 3,317
  9. 2019The Curious Case of Neural Text Degenerationcited by 2,881
  10. 2023MiniGPT-4: Enhancing Vision-Language Understanding with Advanced Large Language Modelscited by 2,689
  11. 2023WebArena: A Realistic Web Environment for Building Autonomous Agentscited by 1,908
  12. 2019Dream to Control: Learning Behaviors by Latent Imaginationcited by 1,598
25 more
  1. 2018World Modelscited by 1,444
  2. 2018Learning Latent Dynamics for Planning from Pixelscited by 1,221
  3. 2023Mind2Web: Towards a Generalist Agent for the Webcited by 1,210
  4. 2022WebShop: Towards Scalable Real-World Web Interaction with Grounded Language Agentscited by 1,197
  5. 2023Mastering Diverse Domains through World Modelscited by 1,193
  6. 2020Mastering Atari with Discrete World Modelscited by 990
  7. 2017Reinforcement Learning with Deep Energy-Based Policiescited by 659
  8. 2024GPT-4V(ision) is a Generalist Web Agent, if Groundedcited by 547
  9. 2023Learning Universal Policies via Text-Guided Video Generationcited by 506
  10. 2023Learning Interactive Real-World Simulatorscited by 440
  11. 2023Language Models can Solve Computer Taskscited by 371
  12. 2023A Real-World WebAgent with Planning, Long Context Understanding, and Program Synthesiscited by 350
  13. 2024WorkArena: How Capable Are Web Agents at Solving Common Knowledge Work Tasks?cited by 348
  14. 2024Agent Workflow Memorycited by 342
  15. 2018Reinforcement Learning on Web Interfaces Using Workflow-Guided Explorationcited by 282
  16. 2019Contrastive Learning of Structured World Modelscited by 206
  17. 2023Synapse: Trajectory-as-Exemplar Prompting with Memory for Computer Controlcited by 180
  18. 2015Action-Conditional Video Prediction using Deep Networks in Atari Gamescited by 171
  19. 2024Tree Search for Language Model Agentscited by 148
  20. 2024Autonomous Evaluation and Refinement of Digital Agentscited by 136
  21. 2017Recurrent Environment Simulatorscited by 68
  22. 2018Probabilistic Recurrent State-Space Modelscited by 64
  23. 2024Can Language Models Serve as Text-Based World Simulators?cited by 50
  24. 2024WILBUR: Adaptive In-Context Learning for Robust and Accurate Web Agentscited by 22
  25. 2024Coffee-Gym: An Environment for Evaluating and Improving Natural Language Feedback on Erroneous Codecited by 9

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 5; 2025: 48; 2026: 63.

116 papers.

Sort:

By area:

  1. 2025MEM1: Learning to Synergize Memory and Reasoning for Efficient Long-Horizon Agentscited by 226
  2. 2025Plan-and-Act: Improving Planning of Agents for Long-Horizon Taskscited by 190
  3. 2025SkillWeaver: Web Agents can Self-Improve by Discovering and Honing Skillscited by 174
  4. 2025WebAgent-R1: Training Web Agents via End-to-End Multi-Turn Reinforcement Learningcited by 162
  5. 2024Understanding World or Predicting Future? A Comprehensive Survey of World Modelscited by 161
  6. 2024Large Language Model-Brained GUI Agents: A Surveycited by 160
  7. 2024Is Your LLM Secretly a World Model of the Internet? Model-Based Planning for Web Agentscited by 144
  8. 2025ReSum: Unlocking Long-Horizon Search Intelligence via Context Summarizationcited by 132
  9. 2024GUI Agents: A Surveycited by 101
  10. 2025UI-Venus Technical Report: Building High-performance UI Agents with RFTcited by 90
  11. 2025A Survey of WebAgents: Towards Next-Generation AI Agents for Web Automation with Large Foundation Modelscited by 87
  12. 2025Agent Learning via Early Experiencecited by 79
38 more
  1. 2025WebEvolver: Enhancing Web Agent Self-Improvement with Coevolving World Modelcited by 56
  2. 2025LLM-based Agents Suffer from Hallucinations: A Survey of Taxonomy, Methods, and Directionscited by 49
  3. 2026Expanding the Capabilities of Reinforcement Learning via Text Feedbackcited by 49
  4. 2026A Survey of Agentic Reasoning for Large Language Models: Towards Recursively Self-Improving and Collective Agentscited by 47
  5. 2025A Survey of State of the Art Large Vision Language Models: Alignment, Benchmark, Evaluations and Challengescited by 46
  6. 2025WMNav: Integrating Vision-Language Models into World Models for Object Goal Navigationcited by 42
  7. 2025Web-Shepherd: Advancing PRMs for Reinforcing Web Agentscited by 41
  8. 2025From Word to World: Can Large Language Models be Implicit Text-based World Models?cited by 40
  9. 2025RLVR-World: Training World Models with Reinforcement Learningcited by 38
  10. 2025MCP Security Bench (MSB): Benchmarking Attacks Against Model Context Protocol in LLM Agentscited by 37
  11. 2025Planning with Reasoning using Vision Language World Modelcited by 36
  12. 2025A Comprehensive Survey of Agents for Computer Use: Foundations, Challenges, and Future Directionscited by 33
  13. 2025Towards Trustworthy GUI Agents: A Surveycited by 33
  14. 2025ViMo: A Generative Visual GUI World Model for App Agentscited by 33
  15. 2025WebChoreArena: Evaluating Web Browsing Agents on Realistic Tedious Web Taskscited by 29
  16. 2026Code2World: A GUI World Model via Renderable Code Generationcited by 26
  17. 2025Symbiotic Cooperation for Web Agents: Harnessing Complementary Strengths of Large and Small LLMscited by 24
  18. 2026WebWorld: A Large-Scale World Model for Web Agent Trainingcited by 23
  19. 2026Qwen-AgentWorld: Language World Models for General Agentscited by 22
  20. 2026MobileDreamer: Generative Sketch World Model for GUI Agentcited by 21
  21. 2025VEM: Environment-Free Exploration for Training GUI Agent with Value Environment Modelcited by 19
  22. 2025WebSynthesis: World-Model-Guided MCTS for Efficient WebUI-Trajectory Synthesiscited by 18
  23. 2025MobileWorldBench: Towards Semantic World Modeling For Mobile Agentscited by 18
  24. 2025A Goal Without a Plan Is Just a Wish: Efficient and Effective Global Planner Training for Long-Horizon Agent Taskscited by 17
  25. 2025LLMs as Scalable, General-Purpose Simulators For Evolving Digital Agent Trainingcited by 17
  26. 2025Test-Time Adaptation for LLM Agents via Environment Interactioncited by 17
  27. 2025When AIs Judge AIs: The Rise of Agent-as-a-Judge Evaluation for LLMscited by 16
  28. 2025WebCoach: Self-Evolving Web Agents with Cross-Session Memory Guidancecited by 15
  29. 2026Current Agents Fail to Leverage World Model as Tool for Foresightcited by 15
  30. 2026Uncertainty Quantification in LLM Agents: Foundations, Emerging Challenges, and Opportunitiescited by 15
  31. 2025StableToolBench-MirrorAPI: Modeling Tool Environments as Mirrors of 7,000+ Real-World APIscited by 14
  32. 2025Dyna-Think: Synergizing Reasoning, Acting, and World Model Simulation in AI Agentscited by 14
  33. 2026Generative Visual Code Mobile World Modelscited by 13
  34. 2026BackdoorAgent: A Unified Framework for Backdoor Attacks on LLM-based Agentscited by 12
  35. 2026SafePred: A Predictive Guardrail for Computer-Using Agents via World Modelscited by 11
  36. 2026Policy and World Modeling Co-Training for Language Agentscited by 11
  37. 2024LLM-Based World Models Can Make Decisions Solely, But Rigorous Evaluations are Neededcited by 10
  38. 2025R-WoM: Retrieval-augmented World Model For Computer-use Agentscited by 10

50 most cited of 116 All 116 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2026World-Model-Augmented Web Agents with Action Correction
  2. 2023A Survey on Large Language Model based Autonomous Agents
  3. 2024Large Language Models Can Self-Improve At Web Agent Tasks
  4. 2025Web World Models
  5. 2026Improving LLM-based Autonomous Web Agents with Filtering
  6. 2025Graph-Augmented Large Language Model Agents: Current Progress and Future Prospects
  7. 2024AutoWebGLM: A Large Language Model-based Web Navigating Agent
  8. 2024Making Large Language Models into World Models with Precondition and Effect Knowledge