mlsanityData to 7 Oct 2026

EARL: Efficient Agentic Reinforcement Learning Systems for Large Language Models

arXiv 2510.05943 ↗First posted 7 Oct 2025cs.DC cs.LGcited by 0

On the map: LLM agents and tool use → Reinforcement learning for LLM agents

Abstract (arXiv)

Reinforcement learning (RL) has become a pivotal component of large language model (LLM) post-training, and agentic RL extends this paradigm to operate as agents through multi-turn interaction and tool use. Scaling such systems exposes two practical bottlenecks: (1) context length grows rapidly during training, inflating memory usage and latency, and triggering out-of-memory (OOM) failures; and (2) intermediate tensors accumulate with context length, making cross-device data movement a major system bottleneck. We present EARL, a scalable system for efficient agentic RL. EARL designs a parallelism selector that dynamically adapts model and training parallelism across RL stages based on sequence length and system load, and a data dispatcher that performs layout-aware, decentralized exchange of intermediate data batches. Together, these components increase throughput, reduce long-context failures, and enable stable large-scale training of agentic LLMs without relying on hard limits or penalties of context length.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Reinforcement learning for LLM agents, one of 16 directions in LLM agents and tool use. Builds on 14 of its 26 references that are in the map. First posted on arXiv 7 Oct 2025.

Builds on

14 of 26 references.

Closest ancestors · 1

Same direction, newest first

  1. 2025Training Long-Context, Multi-Turn Software Engineering Agents with Reinforcement Learning5 Aug

Other references · 13

  1. 2022Training language models to follow instructions with human feedbackcited by 15,712
  2. 2024The Llama 3 Herd of Modelscited by 14,192
  3. 2017Deep reinforcement learning from human preferencescited by 3,846
  4. 2019Megatron-LM: Training Multi-Billion Parameter Language Models Using Model Parallelismcited by 1,829
  5. 2025Kimi k1.5: Scaling Reinforcement Learning with LLMscited by 1,193
  6. 2023SGLang: Efficient Execution of Structured Language Model Programscited by 1,107
  7. 2025L1: Controlling How Long A Reasoning Model Thinks With Reinforcement Learningcited by 381
  8. 2022Reducing Activation Recomputation in Large Transformer Modelscited by 300
  9. 2023DeepSpeed Ulysses: System Optimizations for Enabling Training of Extreme Long Sequence Transformer Modelscited by 239
  10. 2019OpenSpiel: A Framework for Reinforcement Learning in Gamescited by 180
  11. 2025Just Enough Thinking: Efficient Reasoning with Adaptive Length Penalties Reinforcement Learningcited by 51
  12. 2025Agent-Q: Fine-Tuning Large Language Models for Quantum Circuit Generation and Optimizationcited by 17
1 more
  1. 2025QUASAR: Quantum Assembly Code Generation Using Tool-Augmented LLMs via Agentic RLcited by 11

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026Libra: Efficient Resource Management for Agentic RL Post-Training
  2. 2025AReaL: A Large-Scale Asynchronous Reinforcement Learning System for Language Reasoning
  3. 2025AgentRL: Scaling Agentic Reinforcement Learning with a Multi-Turn, Multi-Task Framework
  4. 2026TideRL: Boosting Agentic RL Goodput with Readiness-Aware Scheduling
  5. 2026Scaling Agentic Capabilities, Not Context: Efficient Reinforcement Finetuning for Large Toolspaces
  6. 2026AstraFlow: Dataflow-Oriented Reinforcement Learning for Agentic LLMs
  7. 2025Toward Efficient Exploration by Large Language Model Agents
  8. 2025Tool-R1: Sample-Efficient Reinforcement Learning for Agentic Tool Use