mlsanityData to 7 Oct 2026

WebCoT: Enhancing Web Agent Reasoning by Reconstructing Chain-of-Thought in Reflection, Branching, and Rollback

arXiv 2505.20013 ↗First posted 26 May 2025cs.CLcited by 14

On the map: LLM reasoning → Reasoning for agents, planning and code

also in 1 more
Abstract (arXiv)

Web agents powered by Large Language Models (LLMs) show promise for next-generation AI, but their limited reasoning in uncertain, dynamic web environments hinders robust deployment. In this paper, we identify key reasoning skills essential for effective web agents, i.e., reflection & lookahead, branching, and rollback, and curate trajectory data that exemplifies these abilities by reconstructing the agent's (inference-time) reasoning algorithms into chain-of-thought rationales. We conduct experiments in the agent self-improving benchmark, OpenWebVoyager, and demonstrate that distilling salient reasoning patterns into the backbone LLM via simple fine-tuning can substantially enhance its performance. Our approach yields significant improvements across multiple benchmarks, including WebVoyager, Mind2web-live, and SimpleQA (web search), highlighting the potential of targeted reasoning skill enhancement for web agents.

Key facts. Cited by 14 papers in the map (data through 7 Oct 2026). Placed in Reasoning for agents, planning and code, one of 17 directions in LLM reasoning. Builds on 37 of its 53 references that are in the map. First posted on arXiv 26 May 2025.

Builds on

37 of 53 references.

Closest ancestors · 6

Same direction, newest first

  1. 2025Agentic Reasoning and Tool Integration for LLMs via Reinforcement Learning28 Apr
  2. 2025Agent models: Internalizing Chain-of-Action Generation into Reasoning models9 Mar
  3. 2025Agentic Reasoning: A Streamlined Framework for Enhancing LLM Reasoning with Agentic Tools7 Feb
  4. 2025OverThink: Slowdown Attacks on Reasoning LLMs4 Feb
  5. 2024OpenAI o1 System Card21 Dec
  6. 2024MAGDi: Structured Distillation of Multi-Agent Interaction Graphs Improves Reasoning in Smaller Language Models2 Feb

Other references · 31

  1. 2023GPT-4 Technical Reportcited by 18,132
  2. 2024The Llama 3 Herd of Modelscited by 14,192
  3. 2025DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learningcited by 6,146
  4. 2024Measuring short-form factuality in large language modelscited by 345
  5. 2025WebThinker: Empowering Large Reasoning Models with Deep Research Capabilitycited by 320
  6. 2025Towards Large Reasoning Models: A Survey of Reinforced Reasoning with Large Language Modelscited by 291
  7. 2024WebRL: Training LLM Web Agents via Self-Evolving Online Curriculum Reinforcement Learningcited by 243
  8. 2025WebWalker: Benchmarking LLMs in Web Traversalcited by 171
  9. 2025WebAgent-R1: Training Web Agents via End-to-End Multi-Turn Reinforcement Learningcited by 162
  10. 2024OS-Genesis: Automating GUI Agent Trajectory Construction via Reverse Task Synthesiscited by 155
  11. 2025The Danger of Overthinking: Examining the Reasoning-Action Dilemma in Agentic Taskscited by 150
  12. 2024Is Your LLM Secretly a World Model of the Internet? Model-Based Planning for Web Agentscited by 144
19 more
  1. 2025InfiGUI-R1: Advancing Multimodal GUI Agents from Reactive Actors to Deliberative Reasonerscited by 140
  2. 2024AgentTrek: Agent Trajectory Synthesis via Guiding Replay with Web Tutorialscited by 123
  3. 2024WebCanvas: Benchmarking Web Agents in Online Environmentscited by 119
  4. 2025Between Underthinking and Overthinking: An Empirical Study of Reasoning Length and correctness in LLMscited by 115
  5. 2023ReST meets ReAct: Self-Improvement for Multi-Step Reasoning LLM Agentcited by 78
  6. 2025Reinforcement Learning for Reasoning in Small LLMs: What Works and What Doesn'tcited by 72
  7. 2025Explorer: Scaling Exploration-driven Web Trajectory Synthesis for Multimodal Web Agentscited by 68
  8. 2025WebEvolver: Enhancing Web Agent Self-Improvement with Coevolving World Modelcited by 56
  9. 2024MMInA: Benchmarking Multihop Multimodal Internet Agentscited by 55
  10. 2024Large Language Models Can Self-Improve At Web Agent Taskscited by 54
  11. 2024OpenWebVoyager: Building Multimodal Web Agents via Iterative Real-World Exploration, Feedback and Optimizationcited by 45
  12. 2025InSTA: Towards Internet-Scale Training For Agentscited by 31
  13. 2025BEARCUBS: A benchmark for computer-using web agentscited by 29
  14. 2024Do We Really Need a Complex Agent System? Distill Embodied Agent into a Single Modelcited by 19
  15. 2024Leopard: A Vision Language Model For Text-Rich Multi-Image Taskscited by 18
  16. 2025Beyond 'Aha!': Toward Systematic Meta-Abilities Alignment in Large Reasoning Modelscited by 17
  17. 2025WebRollback: Enhancing Web Agents with Explicit Rollback Mechanismscited by 14
  18. 2024Cognitive Kernel: An Open-source Agent System towards Generalist Autopilotscited by 9
  19. 2025Streaming Looking Ahead with Token-level Self-rewardcited by 2

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 3; 2026: 11.

14 papers.

Sort:

By area:

  1. 2025MobileGUI-RL: Advancing Mobile GUI Agent through Reinforcement Learning in Online Environmentcited by 51
  2. 2025Cognitive Kernel-Pro: A Framework for Deep Research Agents and Agent Foundation Models Trainingcited by 42
  3. 2026Inference-Time Scaling of Verification: Self-Evolving Deep Research Agents via Test-Time Rubric-Guided Verificationcited by 19
  4. 2025WebSynthesis: World-Model-Guided MCTS for Efficient WebUI-Trajectory Synthesiscited by 18
  5. 2026PROGRESSLM: Towards Progress Reasoning in Vision-Language Modelscited by 13
  6. 2026Verified Critical Step Optimization for LLM Agentscited by 13
  7. 2026Marco DeepResearch: Unlocking Efficient Deep Research Agents via Verification-Centric Designcited by 12
  8. 2026Search-R2: Enhancing Search-Integrated Reasoning via Actor-Refiner Collaborationcited by 10
  9. 2026From Question Answering to Task Completion: A Survey on Agent System and Harness Designcited by 9
  10. 2026Faithful Mobile GUI Agents with Guided Advantage Estimatorcited by 1
  11. 2026SciResearcher: Scaling Deep Research Agents for Frontier Scientific Reasoningcited by 1
  12. 2026Rollback the World, Keep the Reflection: Rollback-Induced Reflection for Long-Horizon LLM Agentscited by 1
2 more
  1. 2026CurateEvo: Data-Curation Evolving for Agentic Post-Trainingcited by 0
  2. 2026ProgressCompass: Embodied Progress Reward Models Are Lost Without the Right Contextcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026AI Planning Framework for LLM-Based Web Agents
  2. 2026Ares: Adaptive Reasoning Effort Selection for Efficient LLM Agents
  3. 2025WebSailor: Navigating Super-human Reasoning for Web Agent
  4. 2026WebFactory: Automated Compression of Foundational Language Intelligence into Grounded Web Agents
  5. 2024Textualized Agent-Style Reasoning for Complex Tasks by Multiple Round LLM Generation
  6. 2026DRIVE: Modeling Skills at the Reasoning and Interaction Levels for Web Agents under Continual Learning
  7. 2026Why Do LLM-based Web Agents Fail? A Hierarchical Planning Perspective
  8. 2025Branch-and-Browse: Efficient and Controllable Web Exploration with Tree-Structured Reasoning and Action Memory