Scaling Verifiable Environments for Long-horizon Work Agents
Not on the map. Why?
Abstract (arXiv)
Work agents operate over digital artifacts to execute professional knowledge-intensive work, requiring training environments that support long-horizon interaction and trustworthy verification. However, hand-crafted environments incur prohibitive engineering overhead that prevents environment scaling, whereas synthesis methods sacrifice workspace complexity, realism, or grounded verifiability. To bridge this gap, we introduce WorkForge, a scalable synthesis framework for constructing verifiable work-agent environments from real-world resources. Starting from expert workflows, WorkForge first identifies the resources, decisions, and deliverables required by each workflow. It then retrieves relevant real-world files and organizes them into a workspace. WorkForge inspects the workspace to extract concrete, checkable facts about its content. These factual anchors fix which task types the workspace can support and how their outcomes can be verified. Therefore, WorkForge derives each task's instructions, solution plan, and complementary programmatic and semantic verifiers directly from these factual anchors, keeping verification traceable to observable workspace evidence. Furthermore, we construct 16.7K verifiable environments across 40 professional domains, with workspaces collectively covering 60 file types. Post-training Qwen3.5-35B-A3B-Base improves GDPVal from 45.5 to 73.6 and APEX Score from 5.0 to 21.3, while enabling Qwen3.5-27B to achieve highly competitive performance and outperform strong competitors. Our analyses confirm the efficacy of the proposed method and reveal consistent scaling behaviors across both data volume and interaction horizons.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 28 of its 38 references that are in the map. First posted on arXiv 4 Oct 2026.
Builds on
28 of 38 references.
References · 28
- 2023WebArena: A Realistic Web Environment for Building Autonomous Agentscited by 2,006
- 2026DeepSeek-V4: Towards Highly Efficient Million-Token Context Intelligencecited by 1,040
- 2024AppWorld: A Controllable World of Apps and People for Benchmarking Interactive Coding Agentscited by 401
- 2023MLAgentBench: Evaluating Language Agents on Machine Learning Experimentationcited by 366
- 2024WorkArena: How Capable Are Web Agents at Solving Common Knowledge Work Tasks?cited by 363
- 2025PaperBench: Evaluating AI's Ability to Replicate AI Researchcited by 321
- 2025APIGen-MT: Agentic Pipeline for Multi-Turn Data Generation via Simulated Agent-Human Interplaycited by 167
- 2025GDPval: Evaluating AI Model Performance on Real-World Economically Valuable Taskscited by 158
- 2024WorkArena++: Towards Compositional Planning and Reasoning-based Common Knowledge Work Taskscited by 90
- 2025TOUCAN: Synthesizing 1.5M Tool-Agentic Data from Real-World MCP Environmentscited by 69
- 2026Agent World Model: Infinity Synthetic Environments for Agentic Reinforcement Learningcited by 66
- 2026Agent-World: Scaling Real-World Environment Synthesis for Evolving General Agent Intelligencecited by 54
16 more
- 2025AgentSynth: Scalable Task Generation for Generalist Computer-Use Agentscited by 46
- 2026On Data Engineering for Scaling LLM Terminal Capabilitiescited by 43
- 2026ScaleEnv: Scaling Environment Synthesis from Scratch for Generalist Interactive Tool-Use Agent Trainingcited by 32
- 2026Agents' Last Examcited by 31
- 2026Workspace-Bench 1.0: Benchmarking AI Agents on Workspace Tasks with Large-Scale File Dependenciescited by 29
- 2025Scaling Synthetic Task Generation for Agents via Explorationcited by 23
- 2026Toward Scalable Terminal Task Synthesis via Skill Graphscited by 22
- 2026Terminal-World: Scaling Terminal-Agent Environments via Agent Skillscited by 20
- 2026AgentV-RL: Scaling Reward Modeling with Agentic Verifiercited by 15
- 2026ClawGym: A Scalable Framework for Building Effective Claw Agentscited by 15
- 2026DIVE: Scaling Diversity in Agentic Task Synthesis for Generalizable Tool Usecited by 13
- 2024Multi-Programming Language Sandbox for LLMscited by 10
- 2026FACET: Preserving Source Intent and Executable State in Terminal Task Synthesiscited by 9
- 2026NexForge: Scaling Agent Capabilities through Requirement-Driven Task Synthesis for LLMscited by 7
- 2025Two Minds Better Than One: Collaborative Reward Modeling for LLM Alignmentcited by 6
- 2026AgentGym2: Benchmarking Large Language Model Agents in De-Idealized Real-World Environmentscited by 3
Built on by
Not cited yet.