mlsanityData to 7 Oct 2026

Libra: Efficient Resource Management for Agentic RL Post-Training

arXiv 2606.03077 ↗First posted 2 Jun 2026cs.LG cs.AI cs.DCcited by 2

On the map: LLM agents and tool use → Reinforcement learning for LLM agents

Abstract (arXiv)

Reinforcement learning (RL) has emerged as a standard post-training paradigm for shaping large language models (LLMs) into capable agents. In agentic RL, the rollout stage generates trajectories while invoking tools, producing long-tailed and non-stationary workloads that expose two fundamental challenges. First, due to the long-tailed response distribution, a small fraction of trajectories dominates rollout makespan.Second, rollout and training differ in their compute patterns, memory demands, and sensitivity to sequence length. As the policy evolves, shifts in the workload distribution further change their relative resource demands, making it difficult to maintain balanced execution across the two stages. We present Libra, an adaptive runtime for agentic RL post-training with two complementary components: (1) intra-stage scheduling via a Causality-Guided Bucket Scheduler that routes requests across execution buckets with different parallelism configurations, reducing delays from rollout stragglers; and (2) cross-stage coordination that dynamically reallocates workers between rollout and training as the workload changes. It moves workers between the two stages through a non-blocking protocol without interrupting ongoing training. Evaluated on a 48x NVIDIA A800 GPU cluster and a 160x Ascend 910B3 NPU cluster across three agentic benchmarks, Libra achieves up to 4.2x higher throughput and up to 2.7x faster reward convergence

Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Placed in Reinforcement learning for LLM agents, one of 16 directions in LLM agents and tool use. Builds on 49 of its 67 references that are in the map. First posted on arXiv 2 Jun 2026.

Builds on

49 of 67 references.

Closest ancestors · 7

Same direction, newest first

  1. 2026Heddle: A Distributed Orchestration System for Agentic RL Rollout30 Mar
  2. 2025RollArt: Disaggregated Multi-Task Agentic RL Training at Scale27 Dec
  3. 2025RLinf: Flexible and Efficient Large-scale Reinforcement Learning via Macro-to-Micro Flow Transformation19 Sep
  4. 2025The Landscape of Agentic Reinforcement Learning for LLMs: A Survey2 Sep
  5. 2025VerlTool: Towards Holistic Agentic Reinforcement Learning with Tool Use1 Sep
  6. 2025ReTool: Reinforcement Learning for Strategic Tool Use in LLMs15 Apr
  7. 2023Reflexion: Language Agents with Verbal Reinforcement Learning20 Mar

Other references · 42

  1. 2022Training language models to follow instructions with human feedbackcited by 15,712
  2. 2017Proximal Policy Optimization Algorithmscited by 14,441
  3. 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 8,851
  4. 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,355
  5. 2025Qwen3 Technical Reportcited by 8,195
  6. 2023Efficient Memory Management for Large Language Model Serving with PagedAttentioncited by 5,767
  7. 2017Deep reinforcement learning from human preferencescited by 3,846
  8. 2025Gemini 2.5: Pushing the Frontier with Advanced Reasoning, Multimodality, Long Context, and Next Generation Agentic Capabilitiescited by 3,655
  9. 2022Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedbackcited by 3,634
  10. 2023SWE-bench: Can Language Models Resolve Real-World GitHub Issues?cited by 3,193
  11. 2025DAPO: An Open-Source LLM Reinforcement Learning System at Scalecited by 2,678
  12. 2025Search-R1: Training LLMs to Reason and Leverage Search Engines with Reinforcement Learningcited by 1,371
30 more
  1. 2023PyTorch FSDP: Experiences on Scaling Fully Sharded Data Parallelcited by 641
  2. 2024DistServe: Disaggregating Prefill and Decoding for Goodput-optimized Large Language Model Servingcited by 595
  3. 2024Taming Throughput-Latency Tradeoff in LLM Inference with Sarathi-Servecited by 507
  4. 2025AReaL: A Large-Scale Asynchronous Reinforcement Learning System for Language Reasoningcited by 234
  5. 2025R2E-Gym: Procedural Environments and Hybrid Verifiers for Scaling Open-Weights SWE Agentscited by 140
  6. 2024Vidur: A Large-Scale Simulation Framework For LLM Inferencecited by 115
  7. 2023DeepSpeed-Chat: Easy, Fast and Affordable RLHF Training of ChatGPT-like Models at All Scalescited by 107
  8. 2024DistRL: An Asynchronous Distributed Reinforcement Learning Framework for On-Device Control Agentscited by 68
  9. 2025StreamRL: Scalable, Heterogeneous, and Elastic RL for LLMs with Disaggregated Stream Generationcited by 56
  10. 2022Galvatron: Efficient Transformer Training over Multiple GPUs Using Automatic Parallelismcited by 51
  11. 2025Just Enough Thinking: Efficient Reasoning with Adaptive Length Penalties Reinforcement Learningcited by 51
  12. 2025Continuum: Efficient and Robust Multi-Turn LLM Agent Scheduling with KV Cache Time-to-Livecited by 43
  13. 2024Optimizing RLHF Training for Large Language Models with Stage Fusioncited by 41
  14. 2025LlamaRL: A Distributed Asynchronous Reinforcement Learning Framework for Efficient Large-scale LLM Trainingcited by 37
  15. 2025AsyncFlow: An Asynchronous Streaming RL Framework for Efficient LLM Post-Trainingcited by 37
  16. 2025History Rhymes: Accelerating LLM Reinforcement Learning with RhymeRLcited by 35
  17. 2025Laminar: A Scalable Asynchronous RL Post-Training Frameworkcited by 35
  18. 2024ReaL: Efficient RLHF Training of Large Language Models with Parameter Reallocationcited by 34
  19. 2024Don't Stop Me Now: Embedding Based Scheduling for LLMscited by 34
  20. 2025RollPacker: Mitigating Long-Tail Rollouts for Fast, Synchronous RL Post-Trainingcited by 31
  21. 2022Aryl: An Elastic Cluster Scheduler for Deep Learningcited by 27
  22. 2025Seer: Online Context Learning for Fast Synchronous LLM Reinforcement Learningcited by 21
  23. 2025TokenCake: A KV-Cache-centric Serving Framework for LLM-based Multi-Agent Applicationscited by 16
  24. 2025Sailor: Automating Distributed Training over Dynamic, Heterogeneous, and Geo-distributed Clusterscited by 12
  25. 2025Reinforcement Learning for Out-of-Distribution Reasoning in LLMs: An Empirical Study on Diagnosis-Related Group Codingcited by 7
  26. 2026ROSE: Rollout On Serving GPUs via Cooperative Elasticity for Agentic RLcited by 7
  27. 2026Training LLMs with Fault Tolerant HSDP on 100,000 GPUscited by 6
  28. 2026OrchestrRL: Dynamic Compute and Network Orchestration for Disaggregated RLcited by 4
  29. 2026Frontier: Towards Comprehensive and Accurate LLM Inference Simulationcited by 4
  30. 2024Echo: Simulating Distributed Training At Scalecited by 2

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 2.

2 papers.

  1. 2026Rollout Efficiency in Reinforcement Learning for Reasoning Large Language Models: A Taxonomy and Future Directionscited by 0
  2. 2026QwenGyre: An Elastic Reinforcement Learning Framework for Training xLong-Horizon Agentscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Part II: ROLL Flash -- Accelerating RLVR and Agentic Training with Asynchrony
  2. 2026TideRL: Boosting Agentic RL Goodput with Readiness-Aware Scheduling
  3. 2025EARL: Efficient Agentic Reinforcement Learning Systems for Large Language Models
  4. 2026Single-Rollout Asynchronous Optimization for Agentic Reinforcement Learning
  5. 2026ARL-Tangram: Unleash the Resource Efficiency in Agentic Reinforcement Learning
  6. 2026Scheduling Mixed RL Rollouts Beyond Prefix Locality
  7. 2026Learning at the Right Pace: Adaptive Data Scheduling Improves LLM Reinforcement Learning
  8. 2026SortedRL: Accelerating RL Training for LLMs through Online Length-Aware Scheduling