Recursive Environment Scaling for General Long-Horizon Agents
Not on the map. Why?
Abstract (arXiv)
We study environment scaling as a way to increase agent-benchmark difficulty while holding core task objectives fixed. Our framework, Recursive Environment Scaling (RES), transforms existing benchmark environments while preserving their objectives and completion requirements. A proposer changes information access, resource relations, and execution dependencies; execution-grounded checks validate the resulting candidate, and accepted changes are inherited across rounds. This produces related environments for paired evaluation and training-trajectory collection. Experiments on 294 initial tasks from three benchmarks show lower task success for all six evaluation models, with decreases of 57.14--77.14 percentage points on Claw-Eval-Live. With equal trajectory counts and identical optimization settings, fine-tuning on RES trajectories outperforms original-trajectory SFT in all nine model--benchmark comparisons by 7.00--14.61 percentage points. Ablations identify recursive inheritance and dependency composition as important contributors to increased difficulty. Together, these results support environment scaling for both agent evaluation and training.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 28 of its 29 references that are in the map. First posted on arXiv 12 Oct 2026.
Builds on
28 of 29 references.
References · 28
- 2023SWE-bench: Can Language Models Resolve Real-World GitHub Issues?cited by 3,373
- 2023WebArena: A Realistic Web Environment for Building Autonomous Agentscited by 2,006
- 2024τ-bench: A Benchmark for Tool-Agent-User Interaction in Real-World Domainscited by 1,402
- 2023AgentBench: Evaluating LLMs as Agentscited by 1,267
- 2023Mind2Web: Towards a Generalist Agent for the Webcited by 1,242
- 2024OSWorld: Benchmarking Multimodal Agents for Open-Ended Tasks in Real Computer Environmentscited by 1,195
- 2025τ2-Bench: Evaluating Conversational Agents in a Dual-Control Environmentcited by 627
- 2026Terminal-Bench: Benchmarking Agents on Hard, Realistic Tasks in Command Line Interfacescited by 585
- 2024WebVoyager: Building an End-to-End Web Agent with Large Multimodal Modelscited by 438
- 2024AppWorld: A Controllable World of Apps and People for Benchmarking Interactive Coding Agentscited by 401
- 2024WorkArena: How Capable Are Web Agents at Solving Common Knowledge Work Tasks?cited by 363
- 2024Training Software Engineering Agents and Verifiers with SWE-Gymcited by 336
16 more
- 2024TheAgentCompany: Benchmarking LLM Agents on Consequential Real World Taskscited by 326
- 2024The BrowserGym Ecosystem for Web Agent Researchcited by 161
- 2024WorkArena++: Towards Compositional Planning and Reasoning-based Common Knowledge Work Taskscited by 90
- 2025SPICE: Self-Play In Corpus Environments Improves Reasoningcited by 83
- 2026EnvScaler: Scaling Tool-Interactive Environments for LLM Agent via Programmatic Synthesiscited by 63
- 2026Agent-World: Scaling Real-World Environment Synthesis for Evolving General Agent Intelligencecited by 54
- 2025TaskCraft: Automated Generation of Agentic Taskscited by 51
- 2026ScaleEnv: Scaling Environment Synthesis from Scratch for Generalist Interactive Tool-Use Agent Trainingcited by 32
- 2026Workspace-Bench 1.0: Benchmarking AI Agents on Workspace Tasks with Large-Scale File Dependenciescited by 29
- 2026Gym-Anything: Turn any Software into an Agent Environmentcited by 27
- 2026Recursive Synthesis for Long-Horizon Terminal Taskscited by 19
- 2026DIVE: Scaling Diversity in Agentic Task Synthesis for Generalizable Tool Usecited by 13
- 2026Claw-Eval-Live: A Live Agent Benchmark for Evolving Real-World Workflowscited by 13
- 2026CLI-Universe: Towards Verifiable Task Synthesis Engine for Terminal Agentscited by 12
- 2026SPADE: Self-Play in Adaptive Synthetic Executable Environmentscited by 12
- 2026LOGIGEN: Logic-Driven Generation of Verifiable Agentic Taskscited by 7
Built on by
Not cited yet.