mlsanityData to 14 Oct 2026

Claw-R1: A Step-Level Data Middleware System for Agentic Reinforcement Learning

arXiv 2606.09138 ↗First posted 8 Jun 2026cs.LG cs.CLcited by 0

On the map: LLM agents and tool use → Reinforcement learning for LLM agents

Abstract (arXiv)

Training LLMs to act as capable agents requires learning from multi-turn interactions with environments. However, multi-turn interaction trajectories generated by heterogeneous agent runtimes are difficult to transform into structured, high-quality training data that training engines can readily consume. In this paper, we propose AgentWorld, a data factory that bridges heterogeneous agent runtimes and training engines through a Gateway Server and a Data Pool, reframing LLM agent training around a unified data lifecycle from interaction collection to training consumption. The Gateway Server captures multi-turn interactions through an LLM API proxy, while the Data Pool organizes them into standardized step records for curation and training. An interactive dashboard supports trajectory monitoring, step inspection, and data review. In our demonstration, users integrate ready-to-use white-box and black-box agents, trace and curate their interaction data across the pipeline, and extend the factory to new agents and datasets through lightweight configuration. Quantitative experiments across multiple datasets and RL methods demonstrate training support for different agent runtimes. AgentWorld provides a practical foundation for transforming heterogeneous multi-turn experience into training-ready data for LLM agents. Our code is available at https://github.com/AgentR1/Claw-R1 and the demonstration video can be found at link https://youtu.be/Pw47dAOw6B0.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Reinforcement learning for LLM agents, one of 16 directions in LLM agents and tool use. Builds on 17 of its 24 references that are in the map. First posted on arXiv 8 Jun 2026.

Builds on

17 of 24 references.

Closest ancestors · 6

Same direction, newest first

  1. 2026CAPO: Critic-Guided Action-Aligned Policy Optimization for Advancing LLM Agent Capabilities20 Apr
  2. 2026OpenClaw-RL: Train Any Agent Simply by Talking10 Mar
  3. 2025AutoForge: Automated Environment Synthesis for Agentic Reinforcement Learning28 Dec
  4. 2025VerlTool: Towards Holistic Agentic Reinforcement Learning with Tool Use1 Sep
  5. 2025Agent Lightning: Train ANY AI Agents with Reinforcement Learning5 Aug
  6. 2025Group-in-Group Policy Optimization for LLM Agent Training16 May

Other references · 11

  1. 2022Training language models to follow instructions with human feedbackcited by 15,971
  2. 2017Proximal Policy Optimization Algorithmscited by 15,215
  3. 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 9,379
  4. 2024HybridFlow: A Flexible and Efficient RLHF Frameworkcited by 2,057
  5. 2025Search-R1: Training LLMs to Reason and Leverage Search Engines with Reinforcement Learningcited by 1,444
  6. 2022WebShop: Towards Scalable Real-World Web Interaction with Grounded Language Agentscited by 1,251
  7. 2020ALFWorld: Aligning Text and Embodied Environments for Interactive Learningcited by 1,205
  8. 2025WebShaper: Agentically Data Synthesizing via Information-Seeking Formalizationcited by 121
  9. 2024AgentInstruct: Toward Generative Teaching with Agentic Flowscited by 89
  10. 2025Chain-of-Agents: End-to-End Agent Foundation Models via Multi-Agent Distillation and Agentic RLcited by 80
  11. 2025WebSailor-V2: Bridging the Chasm to Proprietary Agents via Synthetic Data and Scalable Reinforcement Learningcited by 61

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025Agent-R1: A Unified and Modular Framework for Agentic Reinforcement Learning
  2. 2025Reinforcement Learning for Long-Horizon Interactive LLM Agents
  3. 2026EnvCraft: Synthesizing Executable Environments in Agentic RL for Claw-like Agent
  4. 2025AgentGym-RL: Training LLM Agents for Long-Horizon Decision Making through Multi-Turn Reinforcement Learning
  5. 2025MUA-RL: Multi-turn User-interacting Agent Reinforcement Learning for agentic tool use
  6. 2025The Landscape of Agentic Reinforcement Learning for LLMs: A Survey
  7. 2026From Self-Evolving Synthetic Data to Verifiable-Reward RL: Post-Training Multi-turn Interactive Tool-Using Agents
  8. 2026Molt: A Scalable PyTorch-Native Training Framework for Agentic Reinforcement Learning