WebFactory: Automated Compression of Foundational Language Intelligence into Grounded Web Agents
On the map: LLM agents and tool use → Web, search, and research agents
Abstract (arXiv)
Current paradigms for training GUI agents are fundamentally limited by a reliance on either unsafe, non-reproducible live web interactions or costly, scarce human-crafted data and environments. We argue this focus on data volume overlooks a more critical factor: the efficiency of compressing a large language model's (LLM) latent knowledge into actionable agent behavior. We introduce WebFactory, a novel, fully automated closed-loop reinforcement learning pipeline for GUI agents, systematically compressing LLM-encoded internet intelligence into efficient, grounded actions. Our pipeline features a process of scalable environment synthesis, knowledge-aware task generation, LLM-powered trajectory collection, decomposed reward RL training, and systematic agent evaluation. Remarkably, our agent demonstrates exceptional data efficiency and generalization. Trained on synthetic data from only 10 websites within WebFactory, it achieves performance comparable to GUI agents trained on the same amount of human-annotated data from a much larger set of environments. This superior performance is consistent across our internal offline and online transfer benchmarks, where our agent also significantly outperforms the base foundation model. We further provide critical insights into the "embodiment potential" of different LLM foundations, offering a new axis for model evaluation. This work presents a scalable and cost-effective paradigm for transforming passive internet knowledge into active, grounded intelligence, marking a critical step towards general-purpose interactive agents.
Key facts. Cited by 6 papers in the map (data through 7 Oct 2026). Placed in Web, search, and research agents, one of 16 directions in LLM agents and tool use. Builds on 39 of its 44 references that are in the map. First posted on arXiv 5 Mar 2026.
Builds on
39 of 44 references.
Closest ancestors · 17
Same direction, newest first
- 2025Go-Browse: Training Web Agents with Structured Exploration4 Jun
- 2025WebChoreArena: Evaluating Web Browsing Agents on Realistic Tedious Web Tasks2 Jun
- 2025WebAgent-R1: Training Web Agents via End-to-End Multi-Turn Reinforcement Learning22 May
- 2025REAL: Benchmarking Autonomous Agents on Deterministic Simulations of Real Websites15 Apr
- 2025SkillWeaver: Web Agents can Self-Improve by Discovering and Honing Skills9 Apr
- 2025Explorer: Scaling Exploration-driven Web Trajectory Synthesis for Multimodal Web Agents17 Feb
- 2025InSTA: Towards Internet-Scale Training For Agents10 Feb
- 2024Proposer-Agent-Evaluator(PAE): Autonomous Skill Discovery For Foundation Model Internet Agents17 Dec
- 2024AgentTrek: Agent Trajectory Synthesis via Guiding Replay with Web Tutorials12 Dec
- 2024The BrowserGym Ecosystem for Web Agent Research6 Dec
- 2024WebRL: Training LLM Web Agents via Self-Evolving Online Curriculum Reinforcement Learning4 Nov
- 2024AgentOccam: A Simple Yet Strong Baseline for LLM-Based Web Agents17 Oct
5 more
- 2024NNetNav: Unsupervised Learning of Browser Agents Through Environment Interaction in the Wild3 Oct
- 2024WebCanvas: Benchmarking Web Agents in Online Environments18 Jun
- 2024WebVoyager: Building an End-to-End Web Agent with Large Multimodal Models25 Jan
- 2023Mind2Web: Towards a Generalist Agent for the Web9 Jun
- 2022WebShop: Towards Scalable Real-World Web Interaction with Grounded Language Agents4 Jul
Other references · 22
- 2023GPT-4 Technical Reportcited by 18,132
- 2022Training language models to follow instructions with human feedbackcited by 15,712
- 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,355
- 2023Reflexion: Language Agents with Verbal Reinforcement Learningcited by 4,064
- 2017Deep reinforcement learning from human preferencescited by 3,846
- 2023Tree of Thoughts: Deliberate Problem Solving with Large Language Modelscited by 3,155
- 2023Voyager: An Open-Ended Embodied Agent with Large Language Modelscited by 2,081
- 2021Decision Transformer: Reinforcement Learning via Sequence Modelingcited by 1,528
- 2021Offline Reinforcement Learning with Implicit Q-Learningcited by 1,245
- 2020ALFWorld: Aligning Text and Embodied Environments for Interactive Learningcited by 1,143
- 2023Android in the Wild: A Large-Scale Dataset for Android Device Controlcited by 314
- 2018Reinforcement Learning on Web Interfaces Using Workflow-Guided Explorationcited by 282
10 more
- 2025GUI-R1 : A Generalist R1-Style Vision-Language Action Model For GUI Agentscited by 240
- 2018BabyAI: A Platform to Study the Sample Efficiency of Grounded Language Learningcited by 238
- 2024Windows Agent Arena: Evaluating Multi-Modal OS Agents at Scalecited by 212
- 2024GUIOdyssey: A Comprehensive Dataset for Cross-App GUI Navigation on Mobile Devicescited by 190
- 2023Multimodal Web Navigation with Instruction-Finetuned Foundation Modelscited by 159
- 2024OmniACT: A Dataset and Benchmark for Enabling Multimodal Generalist Autonomous Agents for Desktop and Webcited by 156
- 2024GUICourse: From General Vision Language Models to Versatile GUI Agentscited by 112
- 2024Synatra: Turning Indirect Knowledge into Direct Demonstrations for Digital Agents at Scalecited by 43
- 2025OSWorld-Human: Benchmarking the Efficiency of Computer-Use Agentscited by 37
- 2024Harnessing Webpage UIs for Text-Rich Visual Understandingcited by 28
Built on by
Citing papers per year: 2026: 6.
6 papers.
- 2026Qwen-AgentWorld: Language World Models for General Agentscited by 22
- 2026Learn where to Click from Yourself: On-Policy Self-Distillation for GUI Groundingcited by 9
- 2026Fara-1.5: Scalable Learning Environments for Computer Use Agentscited by 5
- 2026Qwen-CUA: Native Computer Use for (almost) Everythingcited by 5
- 2026EvoCUA-1.5: Online Reinforcement Learning for Multi-turn Computer-Use Agentscited by 2
- 2026From Trajectories to Grounded Preferences: Process Preference Synthesis via Interaction Element Graphs for Web PRMscited by 0
- 2026From Trajectories to Grounded Preferences: Process Preference Synthesis via Interaction Element Graphs for Web PRMs26 Sep 2026
- 2026Qwen-CUA: Native Computer Use for (almost) Everything3 Aug 2026
- 2026EvoCUA-1.5: Online Reinforcement Learning for Multi-turn Computer-Use Agents7 Jul 2026
- 2026Qwen-AgentWorld: Language World Models for General Agents23 Jun 2026
- 2026Fara-1.5: Scalable Learning Environments for Computer Use Agents18 Jun 2026
- 2026Learn where to Click from Yourself: On-Policy Self-Distillation for GUI Grounding1 May 2026
Similar, not cited
Close in wording, no citation either way
- 2026InfiniteWeb: Scalable Web Environment Synthesis for GUI Agent Training
- 2024Large Language Models Can Self-Improve At Web Agent Tasks
- 2026WebXSkill: Skill Learning for Autonomous Web Agents
- 2023Exposing Limitations of Language Model Agents in Sequential-Task Compositions on the Web
- 2026WebGym: Scaling Training Environments for Visual Web Agents with Realistic Tasks
- 2025WorkForceAgent-R1: Incentivizing Reasoning Capability in LLM-based Web Agents via Reinforcement Learning
- 2025Synthesizing Agentic Data for Web Agents with Progressive Difficulty Enhancement Mechanisms
- 2025WebEvolver: Enhancing Web Agent Self-Improvement with Coevolving World Model