mlsanityData to 7 Oct 2026

PEARL: Adaptive Prefill-Decode Execution with Elasticity for Agentic Reinforcement Learning

arXiv 2609.35158 ↗First posted 28 Sep 2026cs.AIcited by 0

On the map: LLM agents and tool use → Reinforcement learning for LLM agents

Abstract (arXiv)

Multi-turn rollout dominates the cost of agentic reinforcement learning (RL). Asynchronous execution and elastic GPU resources can accelerate this stage, but adding rollout replicas yields diminishing returns while training GPUs remain idle between updates. We observe that effective resource use also depends on the prefill--decode (PD) configuration. Both the choice between colocation and disaggregation and the optimal PD ratio vary with the workload, making resource scaling and PD configuration interdependent. Exploiting this opportunity requires selecting effective configurations and realizing their benefits within transient resource-availability windows despite reconfiguration costs. We present PEARL, an asynchronous agentic RL system that coordinates external resource elasticity, temporary reuse of idle training GPUs, and adaptive PD execution. PEARL maintains a unified GPU--worker--role state and uses runtime profiles to predict rollout batch completion time, accounting for environment-induced reductions in decode concurrency. It selects the PD mode and ratio under the current GPU budget and translates each decision into an incremental transition plan that minimizes worker and role changes. Cost-aware switching and borrowing policies suppress transitions with insufficient expected benefit while ensuring timely return of training GPUs. Our evaluation show that PEARL achieves 2.17--2.79× the throughput of fixed-resource ROLL across different LLMs. Compared with RLBoost+, throughput improves by up to approximately 26.9% for Qwen3-8B and 36.3% for Qwen3-30B-A3B.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Reinforcement learning for LLM agents, one of 16 directions in LLM agents and tool use. Builds on 28 of its 41 references that are in the map. First posted on arXiv 28 Sep 2026.

Builds on

28 of 41 references.

Closest ancestors · 2

Same direction, newest first

  1. 2025RollArt: Disaggregated Multi-Task Agentic RL Training at Scale27 Dec
  2. 2025RAGEN: Understanding Self-Evolution in LLM Agents via Multi-Turn Reinforcement Learning24 Apr

Other references · 26

  1. 2025Qwen3 Technical Reportcited by 8,195
  2. 2025Gemini 2.5: Pushing the Frontier with Advanced Reasoning, Multimodality, Long Context, and Next Generation Agentic Capabilitiescited by 3,655
  3. 2023SWE-bench: Can Language Models Resolve Real-World GitHub Issues?cited by 3,193
  4. 2019Megatron-LM: Training Multi-Billion Parameter Language Models Using Model Parallelismcited by 1,829
  5. 2023SGLang: Efficient Execution of Structured Language Model Programscited by 1,107
  6. 2023PyTorch FSDP: Experiences on Scaling Fully Sharded Data Parallelcited by 641
  7. 2023Splitwise: Efficient generative LLM inference using phase splittingcited by 536
  8. 2017Ray: A Distributed Framework for Emerging AI Applicationscited by 516
  9. 2023Prompt Cache: Modular Attention Reuse for Low-Latency Inferencecited by 255
  10. 2025SWE-smith: Scaling Data for Software Engineering Agentscited by 255
  11. 2025GUI-R1 : A Generalist R1-Style Vision-Language Action Model For GUI Agentscited by 240
  12. 2024Mooncake: A KVCache-centric Disaggregated Architecture for LLM Servingcited by 193
14 more
  1. 2023SARATHI: Efficient LLM Inference by Piggybacking Decodes with Chunked Prefillscited by 187
  2. 2025UI-R1: Enhancing Efficient Action Prediction of GUI Agents by Reinforcement Learningcited by 166
  3. 2025Agentic Reasoning: A Streamlined Framework for Enhancing LLM Reasoning with Agentic Toolscited by 106
  4. 2025Reinforcement Learning Optimization for Large-Scale Learning: An Efficient and User-Friendly Scaling Librarycited by 94
  5. 2022Bamboo: Making Preemptible Instances Resilient for Affordable Training of Large DNNscited by 49
  6. 2025Laminar: A Scalable Asynchronous RL Post-Training Frameworkcited by 35
  7. 2025INTELLECT-2: A Reasoning Model Trained Through Globally Decentralized Reinforcement Learningcited by 24
  8. 2026ROSE: Rollout On Serving GPUs via Cooperative Elasticity for Agentic RLcited by 7
  9. 2026Efficient Multi-round LLM Inference over Disaggregated Servingcited by 6
  10. 2025DOPD: A Dynamic PD-Disaggregation Architecture for Maximizing Goodput in LLM Inference Servingcited by 5
  11. 2026Not All Prefills Are Equal: PPD Disaggregation for Multi-turn LLM Servingcited by 5
  12. 2026Bidirectional Resource Scheduling for Disaggregated and Asynchronous RL Post-Trainingcited by 5
  13. 2026TensorHub: Scalable and Elastic Weight Transfer for LLM RL Trainingcited by 4
  14. 2025SeamlessFlow: A Trainer Agent Isolation RL Framework Achieving Bubble-Free Pipelines via Tag Schedulingcited by 3

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026WAR: Workload-Aware Rollouts for Synchronous Agentic Reinforcement Learning
  2. 2026TideRL: Boosting Agentic RL Goodput with Readiness-Aware Scheduling
  3. 2026ARL-Tangram: Unleash the Resource Efficiency in Agentic Reinforcement Learning
  4. 2026Libra: Efficient Resource Management for Agentic RL Post-Training
  5. 2026EfficientRollout: System-Aware Self-Speculative Decoding for RL Rollouts
  6. 2026QwenGyre: An Elastic Reinforcement Learning Framework for Training xLong-Horizon Agents
  7. 2026AInfer-PD: Communication-Safe In-Place Prefill-Decode Multiplexing for Distributed MoE Rollouts
  8. 2025SPEC-RL: Accelerating On-Policy Reinforcement Learning with Speculative Rollouts