mlsanityData to 14 Oct 2026

Large Language Models Can Self-Improve At Web Agent Tasks

arXiv 2405.20309 ↗First posted 30 May 2024cs.LG cs.AI cs.CLcited by 55

On the map: LLM agents and tool use → Web, search, and research agents

also in 1 more
Abstract (arXiv)

Training models to act as agents that can effectively navigate and perform actions in a complex environment, such as a web browser, has typically been challenging due to lack of training data. Large language models (LLMs) have recently demonstrated some capability to navigate novel environments as agents in a zero-shot or few-shot fashion, purely guided by natural language instructions as prompts. Recent research has also demonstrated LLMs have the capability to exceed their base performance through self-improvement, i.e. fine-tuning on data generated by the model itself. In this work, we explore the extent to which LLMs can self-improve their performance as agents in long-horizon tasks in a complex environment using the WebArena benchmark. In WebArena, an agent must autonomously navigate and perform actions on web pages to achieve a specified objective. We explore fine-tuning on three distinct synthetic training data mixtures and achieve a 31% improvement in task completion rate over the base model on the WebArena benchmark through a self-improvement procedure. We additionally contribute novel evaluation metrics for assessing the performance, robustness, capabilities, and quality of trajectories of our fine-tuned agent models to a greater degree than simple, aggregate-level benchmark scores currently used to measure self-improvement.

Key facts. Cited by 55 papers in the map (data through 14 Oct 2026). Placed in Web, search, and research agents, one of 16 directions in LLM agents and tool use. Builds on 28 of its 39 references that are in the map. First posted on arXiv 30 May 2024.

Builds on

28 of 39 references.

Closest ancestors · 3

Same direction, newest first

  1. 2024AutoWebGLM: A Large Language Model-based Web Navigating Agent4 Apr
  2. 2023ReST meets ReAct: Self-Improvement for Multi-Step Reasoning LLM Agent15 Dec
  3. 2023WebArena: A Realistic Web Environment for Building Autonomous Agents25 Jul

Other references · 25

  1. 2020Language Models are Few-Shot Learnerscited by 32,419
  2. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,305
  3. 2022Training language models to follow instructions with human feedbackcited by 15,971
  4. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,599
  5. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,380
  6. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,372
  7. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,459
  8. 2015Distilling the Knowledge in a Neural Networkcited by 10,324
  9. 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,170
  10. 2019DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lightercited by 4,307
  11. 2021Finetuned Language Models Are Zero-Shot Learnerscited by 3,780
  12. 2023Self-Refine: Iterative Refinement with Self-Feedbackcited by 3,442
13 more
  1. 2022Self-Instruct: Aligning Language Models with Self-Generated Instructionscited by 2,724
  2. 2024Chatbot Arena: An Open Platform for Evaluating LLMs by Human Preferencecited by 1,184
  3. 2020MPNet: Masked and Permuted Pre-training for Language Understandingcited by 981
  4. 2023PyTorch FSDP: Experiences on Scaling Fully Sharded Data Parallelcited by 660
  5. 2019Be Your Own Teacher: Improve the Performance of Convolutional Neural Networks via Self Distillationcited by 497
  6. 2023Scaling Relationship on Learning Mathematical Reasoning with Large Language Modelscited by 427
  7. 2023Beyond Human Data: Scaling Self-Training for Problem-Solving with Language Modelscited by 349
  8. 2022Large Language Models are Better Reasoners with Self-Verificationcited by 307
  9. 2024A Tale of Tails: Model Collapse as a Change of Scaling Lawscited by 130
  10. 2023RoboCat: A Self-Improving Generalist Agent for Robotic Manipulationcited by 111
  11. 2023SteP: Stacked LLM Policies for Web Actionscited by 66
  12. 2024DataDreamer: A Tool for Synthetic Data Generation and Reproducible LLM Workflowscited by 43
  13. 2021Unsupervised Neural Machine Translation with Generative Language Models Onlycited by 24

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 10; 2025: 29; 2026: 16.

55 papers.

Sort:

By area:

  1. 2024OpenHands: An Open Platform for AI Software Developers as Generalist Agentscited by 1,002
  2. 2025Plan-and-Act: Improving Planning of Agents for Long-Horizon Taskscited by 195
  3. 2025SkillWeaver: Web Agents can Self-Improve by Discovering and Honing Skillscited by 192
  4. 2025Mobile-Agent-v3: Fundamental Agents for GUI Automationcited by 190
  5. 2025WebAgent-R1: Training Web Agents via End-to-End Multi-Turn Reinforcement Learningcited by 165
  6. 2024OS-Genesis: Automating GUI Agent Trajectory Construction via Reverse Task Synthesiscited by 163
  7. 2024Is Your LLM Secretly a World Model of the Internet? Model-Based Planning for Web Agentscited by 154
  8. 2024Tree Search for Language Model Agentscited by 151
  9. 2025Right Question is Already Half the Answer: Fully Unsupervised LLM Reasoning Incentivizationcited by 128
  10. 2025Inducing Programmatic Skills for Agentic Taskscited by 111
  11. 2025AgentRewardBench: Evaluating Automatic Evaluations of Web Agent Trajectoriescited by 108
  12. 2025A Survey of WebAgents: Towards Next-Generation AI Agents for Web Automation with Large Foundation Modelscited by 87
38 more
  1. 2025Agent-R: Training Language Model Agents to Reflect via Iterative Self-Trainingcited by 82
  2. 2024MindSearch: Mimicking Human Minds Elicits Deep AI Searchercited by 66
  3. 2025Deep Research: A Survey of Autonomous Research Agentscited by 66
  4. 2025WebEvolver: Enhancing Web Agent Self-Improvement with Coevolving World Modelcited by 61
  5. 2024NNetNav: Unsupervised Learning of Browser Agents Through Environment Interaction in the Wildcited by 59
  6. 2024Internal Consistency and Self-Feedback in Large Language Models: A Surveycited by 50
  7. 2026A Survey of Agentic Reasoning for Large Language Models: Towards Recursively Self-Improving and Collective Agentscited by 50
  8. 2024OpenWebVoyager: Building Multimodal Web Agents via Iterative Real-World Exploration, Feedback and Optimizationcited by 46
  9. 2024VideoWebArena: Evaluating Long Context Multimodal Agents with Video Understanding Web Taskscited by 43
  10. 2025OS Agents: A Survey on MLLM-based Agents for General Computing Devices Usecited by 35
  11. 2025InSTA: Towards Internet-Scale Training For Agentscited by 32
  12. 2025Symbiotic Cooperation for Web Agents: Harnessing Complementary Strengths of Large and Small LLMscited by 24
  13. 2025Morae: Proactively Pausing UI Agents for User Choicescited by 19
  14. 2026Self-Improvements in Modern Agentic Systems: A Surveycited by 19
  15. 2025Efficient Agent Training for Computer Usecited by 18
  16. 2025Build the web for agents, not agents for the webcited by 18
  17. 2025WebCoT: Enhancing Web Agent Reasoning by Reconstructing Chain-of-Thought in Reflection, Branching, and Rollbackcited by 17
  18. 2025WebCoach: Self-Evolving Web Agents with Cross-Session Memory Guidancecited by 16
  19. 2025Alignment and Safety in Large Language Models: Safety Mechanisms, Training Paradigms, and Emerging Challengescited by 15
  20. 2024From Imitation to Introspection: Probing Self-Consciousness in Language Modelscited by 14
  21. 2026Agentic Artificial Intelligence (AI): Architectures, Taxonomies, and Evaluation of Large Language Model Agentscited by 14
  22. 2025CuES: A Curiosity-driven and Environment-grounded Synthesis Framework for Agentic RLcited by 10
  23. 2026Skill Self-Play: Pushing the Frontier of LLM Capability with Co-Evolving Skillscited by 10
  24. 2026HATS: Hardness-Aware Trajectory Synthesis for GUI Agentscited by 9
  25. 2026Why Do LLM-based Web Agents Fail? A Hierarchical Planning Perspectivecited by 9
  26. 2026Agentic Environment Engineering for Large Language Models: A Survey of Environment Modeling, Synthesis, Evaluation, and Applicationcited by 8
  27. 2025Self-Improving Loops for Visual Robotic Planningcited by 7
  28. 2026From Chatbot to Digital Colleague: The Paradigm Shift Toward Persistent Autonomous AIcited by 6
  29. 2025Language Models can Self-Improve at State-Value Estimation for Better Searchcited by 4
  30. 2025AgentTypo: Adaptive Typographic Prompt Injection Attacks against Black-box Multimodal Agentscited by 4
  31. 2025Towards Understanding, Analyzing, and Optimizing Agentic AI Execution: A CPU-Centric Perspectivecited by 4
  32. 2025Latent Principle Discovery for Language Model Self-Improvementcited by 2
  33. 2026ST-Lite: Training-Free KV Cache Compression with Spatio-Trajectory Guidance for Long-Horizon GUI Agentscited by 2
  34. 2026Gated-BEPO: Confidence-Gated Bellman Credit Assignment for Large Language Model Agentscited by 2
  35. 2025On the Convergence of Moral Self-Correction in Large Language Modelscited by 1
  36. 2025Compute-Accuracy Pareto Frontiers for Open-Source Reasoning Large Language Modelscited by 1
  37. 2026APEX-Searcher: Refining Credit Assignment with Subgoaling for Agentic Retrieval-Augmented Generationcited by 1
  38. 2026Training Needs Trustworthy Worlds: Verified Synthetic Web Environments for Agent Learningcited by 1

50 most cited of 55 All 55 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2026Improving LLM-based Autonomous Web Agents with Filtering
  2. 2026WebFactory: Automated Compression of Foundational Language Intelligence into Grounded Web Agents
  3. 2026WebXSkill: Skill Learning for Autonomous Web Agents
  4. 2023Exposing Limitations of Language Model Agents in Sequential-Task Compositions on the Web
  5. 2024Web Agents with World Models: Learning and Leveraging Environment Dynamics in Web Navigation
  6. 2023A Real-World WebAgent with Planning, Long Context Understanding, and Program Synthesis
  7. 2025Contextual Experience Replay for Self-Improvement of Language Agents
  8. 2025REAL: Benchmarking Autonomous Agents on Deterministic Simulations of Real Websites