mlsanityData to 7 Oct 2026

WebFactory: Automated Compression of Foundational Language Intelligence into Grounded Web Agents

arXiv 2603.05044 ↗First posted 5 Mar 2026cs.AIcited by 6

On the map: LLM agents and tool use → Web, search, and research agents

Abstract (arXiv)

Current paradigms for training GUI agents are fundamentally limited by a reliance on either unsafe, non-reproducible live web interactions or costly, scarce human-crafted data and environments. We argue this focus on data volume overlooks a more critical factor: the efficiency of compressing a large language model's (LLM) latent knowledge into actionable agent behavior. We introduce WebFactory, a novel, fully automated closed-loop reinforcement learning pipeline for GUI agents, systematically compressing LLM-encoded internet intelligence into efficient, grounded actions. Our pipeline features a process of scalable environment synthesis, knowledge-aware task generation, LLM-powered trajectory collection, decomposed reward RL training, and systematic agent evaluation. Remarkably, our agent demonstrates exceptional data efficiency and generalization. Trained on synthetic data from only 10 websites within WebFactory, it achieves performance comparable to GUI agents trained on the same amount of human-annotated data from a much larger set of environments. This superior performance is consistent across our internal offline and online transfer benchmarks, where our agent also significantly outperforms the base foundation model. We further provide critical insights into the "embodiment potential" of different LLM foundations, offering a new axis for model evaluation. This work presents a scalable and cost-effective paradigm for transforming passive internet knowledge into active, grounded intelligence, marking a critical step towards general-purpose interactive agents.

Key facts. Cited by 6 papers in the map (data through 7 Oct 2026). Placed in Web, search, and research agents, one of 16 directions in LLM agents and tool use. Builds on 39 of its 44 references that are in the map. First posted on arXiv 5 Mar 2026.

Builds on

39 of 44 references.

Closest ancestors · 17

Same direction, newest first

  1. 2025Go-Browse: Training Web Agents with Structured Exploration4 Jun
  2. 2025WebChoreArena: Evaluating Web Browsing Agents on Realistic Tedious Web Tasks2 Jun
  3. 2025WebAgent-R1: Training Web Agents via End-to-End Multi-Turn Reinforcement Learning22 May
  4. 2025REAL: Benchmarking Autonomous Agents on Deterministic Simulations of Real Websites15 Apr
  5. 2025SkillWeaver: Web Agents can Self-Improve by Discovering and Honing Skills9 Apr
  6. 2025Explorer: Scaling Exploration-driven Web Trajectory Synthesis for Multimodal Web Agents17 Feb
  7. 2025InSTA: Towards Internet-Scale Training For Agents10 Feb
  8. 2024Proposer-Agent-Evaluator(PAE): Autonomous Skill Discovery For Foundation Model Internet Agents17 Dec
  9. 2024AgentTrek: Agent Trajectory Synthesis via Guiding Replay with Web Tutorials12 Dec
  10. 2024The BrowserGym Ecosystem for Web Agent Research6 Dec
  11. 2024WebRL: Training LLM Web Agents via Self-Evolving Online Curriculum Reinforcement Learning4 Nov
  12. 2024AgentOccam: A Simple Yet Strong Baseline for LLM-Based Web Agents17 Oct
5 more
  1. 2024NNetNav: Unsupervised Learning of Browser Agents Through Environment Interaction in the Wild3 Oct
  2. 2024WebCanvas: Benchmarking Web Agents in Online Environments18 Jun
  3. 2024WebVoyager: Building an End-to-End Web Agent with Large Multimodal Models25 Jan
  4. 2023Mind2Web: Towards a Generalist Agent for the Web9 Jun
  5. 2022WebShop: Towards Scalable Real-World Web Interaction with Grounded Language Agents4 Jul

Other references · 22

  1. 2023GPT-4 Technical Reportcited by 18,132
  2. 2022Training language models to follow instructions with human feedbackcited by 15,712
  3. 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,355
  4. 2023Reflexion: Language Agents with Verbal Reinforcement Learningcited by 4,064
  5. 2017Deep reinforcement learning from human preferencescited by 3,846
  6. 2023Tree of Thoughts: Deliberate Problem Solving with Large Language Modelscited by 3,155
  7. 2023Voyager: An Open-Ended Embodied Agent with Large Language Modelscited by 2,081
  8. 2021Decision Transformer: Reinforcement Learning via Sequence Modelingcited by 1,528
  9. 2021Offline Reinforcement Learning with Implicit Q-Learningcited by 1,245
  10. 2020ALFWorld: Aligning Text and Embodied Environments for Interactive Learningcited by 1,143
  11. 2023Android in the Wild: A Large-Scale Dataset for Android Device Controlcited by 314
  12. 2018Reinforcement Learning on Web Interfaces Using Workflow-Guided Explorationcited by 282
10 more
  1. 2025GUI-R1 : A Generalist R1-Style Vision-Language Action Model For GUI Agentscited by 240
  2. 2018BabyAI: A Platform to Study the Sample Efficiency of Grounded Language Learningcited by 238
  3. 2024Windows Agent Arena: Evaluating Multi-Modal OS Agents at Scalecited by 212
  4. 2024GUIOdyssey: A Comprehensive Dataset for Cross-App GUI Navigation on Mobile Devicescited by 190
  5. 2023Multimodal Web Navigation with Instruction-Finetuned Foundation Modelscited by 159
  6. 2024OmniACT: A Dataset and Benchmark for Enabling Multimodal Generalist Autonomous Agents for Desktop and Webcited by 156
  7. 2024GUICourse: From General Vision Language Models to Versatile GUI Agentscited by 112
  8. 2024Synatra: Turning Indirect Knowledge into Direct Demonstrations for Digital Agents at Scalecited by 43
  9. 2025OSWorld-Human: Benchmarking the Efficiency of Computer-Use Agentscited by 37
  10. 2024Harnessing Webpage UIs for Text-Rich Visual Understandingcited by 28

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 6.

6 papers.

Similar, not cited

Close in wording, no citation either way

  1. 2026InfiniteWeb: Scalable Web Environment Synthesis for GUI Agent Training
  2. 2024Large Language Models Can Self-Improve At Web Agent Tasks
  3. 2026WebXSkill: Skill Learning for Autonomous Web Agents
  4. 2023Exposing Limitations of Language Model Agents in Sequential-Task Compositions on the Web
  5. 2026WebGym: Scaling Training Environments for Visual Web Agents with Realistic Tasks
  6. 2025WorkForceAgent-R1: Incentivizing Reasoning Capability in LLM-based Web Agents via Reinforcement Learning
  7. 2025Synthesizing Agentic Data for Web Agents with Progressive Difficulty Enhancement Mechanisms
  8. 2025WebEvolver: Enhancing Web Agent Self-Improvement with Coevolving World Model