mlsanityData to 7 Oct 2026

ARL-Tangram: Unleash the Resource Efficiency in Agentic Reinforcement Learning

arXiv 2603.13019 ↗First posted 13 Mar 2026cs.DC cs.AI cs.LGcited by 0

On the map: LLM agents and tool use → Reinforcement learning for LLM agents

Abstract (arXiv)

Agentic reinforcement learning (RL) has emerged as a transformative workload in cloud clusters, enabling large language models (LLMs) to solve complex problems through interactions with real world. However, unlike traditional RL, agentic RL demands substantial external cloud resources, e.g., CPUs for code execution and GPUs for reward models, that exist outside the primary training cluster. Existing agentic RL framework typically rely on static over-provisioning, i.e., resources are often tied to long-lived trajectories or isolated by tasks, which leads to severe resource inefficiency. We propose the action-level orchestration, and incorporate it into ARL-Tangram, a unified resource management system that enables fine-grained external resource sharing and elasticity. ARL-Tangram utilizes a unified action-level formulation and an elastic scheduling algorithm to minimize action completion time (ACT) while satisfying heterogeneous resource constraints. Further, heterogeneous resource managers are tailored to efficiently support the action-level execution on resources with heterogeneous characteristics and topologies. Evaluation on real-world agentic RL tasks demonstrates that ARL-Tangram improves average ACT by up to 4.3×, speeds up the step duration of RL training by up to 1.5×, and saves the external resources by up to 71.2%. This system has been deployed to support the training of the MiMo series models.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Reinforcement learning for LLM agents, one of 16 directions in LLM agents and tool use. Builds on 38 of its 54 references that are in the map. First posted on arXiv 13 Mar 2026.

Builds on

38 of 54 references.

Closest ancestors · 4

Same direction, newest first

  1. 2025Let It Flow: Agentic Crafting on Rock and Roll, Building the ROME Model within an Open Agentic Learning Ecosystem31 Dec
  2. 2025SkyRL-Agent: Efficient RL Training for Multi-turn LLM Agent20 Nov
  3. 2025VerlTool: Towards Holistic Agentic Reinforcement Learning with Tool Use1 Sep
  4. 2023Voyager: An Open-Ended Embodied Agent with Large Language Models25 May

Other references · 34

  1. 2022Training language models to follow instructions with human feedbackcited by 15,712
  2. 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 8,851
  3. 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,355
  4. 2025Qwen3 Technical Reportcited by 8,195
  5. 2023SWE-bench: Can Language Models Resolve Real-World GitHub Issues?cited by 3,193
  6. 2023RT-2: Vision-Language-Action Models Transfer Web Knowledge to Robotic Controlcited by 2,993
  7. 2022Constitutional AI: Harmlessness from AI Feedbackcited by 2,950
  8. 2024HybridFlow: A Flexible and Efficient RLHF Frameworkcited by 1,980
  9. 2025Search-R1: Training LLMs to Reason and Leverage Search Engines with Reinforcement Learningcited by 1,371
  10. 2025gpt-oss-120b & gpt-oss-20b Model Cardcited by 1,258
  11. 2023SGLang: Efficient Execution of Structured Language Model Programscited by 1,107
  12. 2025BrowseComp: A Simple Yet Challenging Benchmark for Browsing Agentscited by 616
22 more
  1. 2022CodeRL: Mastering Code Generation through Pretrained Models and Deep Reinforcement Learningcited by 342
  2. 2025DeepResearcher: Scaling Deep Research via Reinforcement Learning in Real-world Environmentscited by 268
  3. 2024WebRL: Training LLM Web Agents via Self-Evolving Online Curriculum Reinforcement Learningcited by 243
  4. 2025AReaL: A Large-Scale Asynchronous Reinforcement Learning System for Language Reasoningcited by 234
  5. 2026MiMo-V2-Flash Technical Reportcited by 163
  6. 2023DeepSpeed-Chat: Easy, Fast and Affordable RLHF Training of ChatGPT-like Models at All Scalescited by 107
  7. 2024ServerlessLLM: Low-Latency Serverless Inference for Large Language Modelscited by 78
  8. 2024StepCoder: Improve Code Generation with Reinforcement Learning from Compiler Feedbackcited by 74
  9. 2025CUDA-L1: Improving CUDA Optimization via Contrastive Reinforcement Learningcited by 59
  10. 2025StreamRL: Scalable, Heterogeneous, and Elastic RL for LLMs with Disaggregated Stream Generationcited by 56
  11. 2024Optimizing RLHF Training for Large Language Models with Stage Fusioncited by 41
  12. 2024NeMo-Aligner: Scalable Toolkit for Efficient Model Alignmentcited by 40
  13. 2025AsyncFlow: An Asynchronous Streaming RL Framework for Efficient LLM Post-Trainingcited by 37
  14. 2025History Rhymes: Accelerating LLM Reinforcement Learning with RhymeRLcited by 35
  15. 2025RollPacker: Mitigating Long-Tail Rollouts for Fast, Synchronous RL Post-Trainingcited by 31
  16. 2024BLITZSCALE: Fast and Live Large Model Autoscaling with O(1) Host Cachingcited by 23
  17. 2022Autothrottle: A Practical Bi-Level Approach to Resource Management for SLO-Targeted Microservicescited by 12
  18. 2025HydraServe: Minimizing Cold Start Latency for Serverless LLM Serving in Public Cloudscited by 12
  19. 2025DistFlow: A Fully Distributed RL Framework for Scalable and Efficient LLM Post-Trainingcited by 9
  20. 2026MegaFlow: Large-Scale Distributed Orchestration System for the Agentic Eracited by 9
  21. 2025λScale: Enabling Fast Scaling for Serverless Large Language Model Inferencecited by 7
  22. 2025SeaLLM: Service-Aware and Latency-Optimized Resource Sharing for Large Language Model Inferencecited by 4

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026Libra: Efficient Resource Management for Agentic RL Post-Training
  2. 2026PEARL: Adaptive Prefill-Decode Execution with Elasticity for Agentic Reinforcement Learning
  3. 2025The Landscape of Agentic Reinforcement Learning for LLMs: A Survey
  4. 2026TideRL: Boosting Agentic RL Goodput with Readiness-Aware Scheduling
  5. 2025RollArt: Disaggregated Multi-Task Agentic RL Training at Scale
  6. 2026AstraFlow: Dataflow-Oriented Reinforcement Learning for Agentic LLMs
  7. 2026AgentJet: A Distributed Swarm Training Framework for Agentic Reinforcement Learning
  8. 2025EARL: Efficient Agentic Reinforcement Learning Systems for Large Language Models