mlsanityData to 7 Oct 2026

EnvCraft: Synthesizing Executable Environments in Agentic RL for Claw-like Agent

arXiv 2609.05576 ↗First posted 4 Sep 2026cs.AI cs.CLcited by 1

On the map: LLM agents and tool use → Reinforcement learning for LLM agents

Abstract (arXiv)

The paradigm of LLMs has rapidly shifted from passive language interfaces to autonomous Claw-like agents that execute long-horizon tasks across stateful workspaces. While Agentic Reinforcement Learning (Agentic RL) provides a promising path to optimize these agents, its scaling is heavily bottlenecked by the severe scarcity of interactive training environments. Existing synthetic environments are strictly limited to tool-calling endpoints, rendering them insufficient for accommodating the end-to-end real-world demands of claw-like agents. To bridge this gap, we introduce EnvCraft, an automated framework for synthesizing executable environments and scalable training data. Specifically, EnvCraft employs an environment synthesis engine to build sandbox-isolated workspaces, alongside a topology-aware data generation engine to produce coherent task trajectories. Overall, we synthesize 139 interactive environments comprising approximately 20K complex tasks for Agentic RL training. Experiments on Qwen3/3.5 models (8B-32B) show that our method yields gains of up to +11.9% on Claw-style benchmarks and +8.0% on general tool-use benchmarks, with concurrent reductions in inference token cost. The results confirm that synthesized executable environments provide robust and generalizable learning signals for training.

Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Placed in Reinforcement learning for LLM agents, one of 16 directions in LLM agents and tool use. Builds on 25 of its 45 references that are in the map. First posted on arXiv 4 Sep 2026.

Builds on

25 of 45 references.

Closest ancestors · 6

Same direction, newest first

  1. 2025Simulating Environments with Reasoning Models for Agent Training3 Nov
  2. 2025Tool Zero: Training Tool-Augmented LLMs via Pure RL from Scratch2 Nov
  3. 2025SimpleTIR: End-to-End Reinforcement Learning for Multi-Turn Tool-Integrated Reasoning2 Sep
  4. 2025ToolRL: Reward is All Tool Learning Needs16 Apr
  5. 2025ReTool: Reinforcement Learning for Strategic Tool Use in LLMs15 Apr
  6. 2025ToRL: Scaling Tool-Integrated RL30 Mar

Other references · 19

  1. 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 8,851
  2. 2025DAPO: An Open-Source LLM Reinforcement Learning System at Scalecited by 2,678
  3. 2024HybridFlow: A Flexible and Efficient RLHF Frameworkcited by 1,980
  4. 2023ToolLLM: Facilitating Large Language Models to Master 16000+ Real-world APIscited by 1,880
  5. 2025Search-R1: Training LLMs to Reason and Leverage Search Engines with Reinforcement Learningcited by 1,371
  6. 2024τ-bench: A Benchmark for Tool-Agent-User Interaction in Real-World Domainscited by 1,280
  7. 2024Scaling Synthetic Data Creation with 1,000,000,000 Personascited by 397
  8. 2025WebSailor: Navigating Super-human Reasoning for Web Agentcited by 218
  9. 2025ACEBench: Who Wins the Match Point in Tool Usage?cited by 66
  10. 2025TOUCAN: Synthesizing 1.5M Tool-Agentic Data from Real-World MCP Environmentscited by 61
  11. 2025Towards General Agentic Intelligence via Environment Scalingcited by 51
  12. 2026Agent World Model: Infinity Synthetic Environments for Agentic Reinforcement Learningcited by 51
7 more
  1. 2024Can Language Models Serve as Text-Based World Simulators?cited by 50
  2. 2026EnvScaler: Scaling Tool-Interactive Environments for LLM Agent via Programmatic Synthesiscited by 46
  3. 2025Scaling Agent Learning via Experience Synthesiscited by 40
  4. 2025From Word to World: Can Large Language Models be Implicit Text-based World Models?cited by 40
  5. 2026ScaleEnv: Scaling Environment Synthesis from Scratch for Generalist Interactive Tool-Use Agent Trainingcited by 23
  6. 2025Scaling Generalist Data-Analytic Agentscited by 11
  7. 2026EnvFactory: Scaling Tool-Use Agents via Executable Environments Synthesis and Robust RLcited by 11

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 1.

1 paper.

Similar, not cited

Close in wording, no citation either way

  1. 2025CuES: A Curiosity-driven and Environment-grounded Synthesis Framework for Agentic RL
  2. 2026Claw-R1: A Step-Level Data Middleware System for Agentic Reinforcement Learning
  3. 2025AutoForge: Automated Environment Synthesis for Agentic Reinforcement Learning
  4. 2026ClawEnvKit: Automatic Environment Generation for Claw-Like Agents
  5. 2026EnvHarness: Awakening Static Worlds for Agent Learning
  6. 2026Mock Worlds, Real Skills: Building Small Agentic Language Models with Synthetic Tasks, Simulated Environments, and Rubric-Based Rewards
  7. 2025Generalizable End-to-End Tool-Use RL with Synthetic CodeGym
  8. 2026Agent-World: Scaling Real-World Environment Synthesis for Evolving General Agent Intelligence