mlsanityData to 7 Oct 2026

MCP-Universe RL: A Framework for Training MCP Tool-Use Agents via Reinforcement Learning

arXiv 2608.22167 ↗First posted 23 Aug 2026cs.AI cs.LGcited by 0

On the map: LLM agents and tool use → Reinforcement learning for LLM agents

Abstract (arXiv)

Reinforcement learning (RL) has become an effective way to improve the tool-use ability of large language models (LLMs), but most existing RL frameworks stop at the policy update. For every new domain, the user is left with two hard systems problems: standing up an isolated environment for each of hundreds of concurrent trajectories and connecting it to training, and scheduling the rollout so that the GPU stays busy across long, multi-turn episodes that spend much of their time stalled on slow tool calls. We present MCP-Universe RL (MCP-U RL), an open-source framework that takes over both. It uses the Model Context Protocol (MCP) as the interface to the environment, so any tool already exposed as an MCP server plugs into training with no RL-specific integration code. It builds the two missing layers once and reuses them across domains: an environment-orchestration layer that provisions, isolates, and recycles the MCP environments over a pluggable container backend, and a rollout-orchestration layer whose staged pipeline overlaps trajectories to keep the GPU busy while episodes wait on tools. A backend-agnostic training layer then applies the update through an existing RL backend, with veRL and slime integrations. With one configuration, changing only the task specification, we train software-engineering, deep-research, and general tool-use agents on gpt-oss-20b and improve task reward in all three.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Reinforcement learning for LLM agents, one of 16 directions in LLM agents and tool use. Builds on 12 of its 22 references that are in the map. First posted on arXiv 23 Aug 2026.

Builds on

12 of 22 references.

Closest ancestors · 1

Same direction, newest first

  1. 2025Agent Lightning: Train ANY AI Agents with Reinforcement Learning5 Aug

Other references · 11

  1. 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 8,851
  2. 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,355
  3. 2023SWE-bench: Can Language Models Resolve Real-World GitHub Issues?cited by 3,193
  4. 2025DAPO: An Open-Source LLM Reinforcement Learning System at Scalecited by 2,678
  5. 2024HybridFlow: A Flexible and Efficient RLHF Frameworkcited by 1,980
  6. 2025AReaL: A Large-Scale Asynchronous Reinforcement Learning System for Language Reasoningcited by 234
  7. 2024The BrowserGym Ecosystem for Web Agent Researchcited by 152
  8. 2025R2E-Gym: Procedural Environments and Hybrid Verifiers for Scaling Open-Weights SWE Agentscited by 140
  9. 2025MCP-Universe: Benchmarking Large Language Models with Real-World Model Context Protocol Serverscited by 89
  10. 2026Agent World Model: Infinity Synthetic Environments for Agentic Reinforcement Learningcited by 51
  11. 2025MCP-Flow: Facilitating LLM Agents to Master Real-World, Diverse and Scaling MCP Toolscited by 9

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025Agent-R1: A Unified and Modular Framework for Agentic Reinforcement Learning
  2. 2026Demystifying Reinforcement Learning for Long-Horizon Tool-Using Agents: A Comprehensive Recipe
  3. 2025RLVE: Scaling Up Reinforcement Learning for Language Models with Adaptive Verifiable Environments
  4. 2025RLFactory: A Plug-and-Play Reinforcement Learning Post-Training Framework for LLM Multi-Turn Tool-Use
  5. 2025Tool-R1: Sample-Efficient Reinforcement Learning for Agentic Tool Use
  6. 2025Feedback-Driven Tool-Use Improvements in Large Language Models via Automated Build Environments
  7. 2026On Effectiveness and Efficiency of Agentic Tool-calling and RL Training
  8. 2025The Landscape of Agentic Reinforcement Learning for LLMs: A Survey