mlsanityData to 14 Oct 2026

SkyRL-Agent: Efficient RL Training for Multi-turn LLM Agent

arXiv 2511.16108 ↗First posted 20 Nov 2025cs.AIcited by 48

On the map: LLM agents and tool use → Reinforcement learning for LLM agents

Abstract (arXiv)

We introduce SkyRL-Agent, a framework for efficient, multi-turn, long-horizon agent training and evaluation. It provides efficient asynchronous dispatching, lightweight tool integration, and flexible backend interoperability, enabling seamless use with existing RL frameworks such as SkyRL-train, VeRL, and Tinker. Using SkyRL-Agent, we train SA-SWE-32B, a software engineering agent trained from Qwen3-32B (24.4% Pass@1) purely with reinforcement learning. We introduce two key components: an optimized asynchronous pipeline dispatcher that achieves a 1.55x speedup over naive asynchronous batching, and a tool-enhanced training recipe leveraging an AST-based search tool to facilitate code navigation, boost rollout Pass@K, and improve training efficiency. Together, these optimizations enable SA-SWE-32B to reach 39.4% Pass@1 on SWE-Bench Verified with more than 2x cost reduction compared to prior models reaching similar performance. Despite being trained solely on SWE tasks, SA-SWE-32B generalizes effectively to other agentic tasks, including Terminal-Bench, BrowseComp-Plus, and WebArena. We further demonstrate SkyRL-Agent's extensibility through case studies on deep research, computer use, and memory agents, each trained using a different training backend.

Key facts. Cited by 48 papers in the map (data through 14 Oct 2026). Placed in Reinforcement learning for LLM agents, one of 16 directions in LLM agents and tool use. Builds on 39 of its 56 references that are in the map. First posted on arXiv 20 Nov 2025.

Builds on

39 of 56 references.

Closest ancestors · 4

Same direction, newest first

  1. 2025GEM: A Gym for Agentic LLMs1 Oct
  2. 2025VerlTool: Towards Holistic Agentic Reinforcement Learning with Tool Use1 Sep
  3. 2025Agent Lightning: Train ANY AI Agents with Reinforcement Learning5 Aug
  4. 2025ToRL: Scaling Tool-Integrated RL30 Mar

Other references · 35

  1. 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 9,379
  2. 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,695
  3. 2025DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learningcited by 6,330
  4. 2023SWE-bench: Can Language Models Resolve Real-World GitHub Issues?cited by 3,373
  5. 2025DAPO: An Open-Source LLM Reinforcement Learning System at Scalecited by 2,810
  6. 2024HybridFlow: A Flexible and Efficient RLHF Frameworkcited by 2,057
  7. 2023WebArena: A Realistic Web Environment for Building Autonomous Agentscited by 2,006
  8. 2024SWE-agent: Agent-Computer Interfaces Enable Automated Software Engineeringcited by 1,737
  9. 2025Search-R1: Training LLMs to Reason and Leverage Search Engines with Reinforcement Learningcited by 1,444
  10. 2023MemGPT: Towards LLMs as Operating Systemscited by 1,389
  11. 2024RULER: What's the Real Context Size of Your Long-Context Language Models?cited by 1,346
  12. 2025Understanding R1-Zero-Like Training: A Critical Perspectivecited by 1,345
23 more
  1. 2025Qwen3 Embedding: Advancing Text Embedding and Reranking Through Foundation Modelscited by 1,261
  2. 2024OpenHands: An Open Platform for AI Software Developers as Generalist Agentscited by 1,002
  3. 2024Executable Code Actions Elicit Better LLM Agentscited by 564
  4. 2017Ray: A Distributed Framework for Emerging AI Applicationscited by 527
  5. 2025Open-Reasoner-Zero: An Open Source Approach to Scaling Up Reinforcement Learning on the Base Modelcited by 461
  6. 2024Agentless: Demystifying LLM-based Software Engineering Agentscited by 421
  7. 2024Training Software Engineering Agents and Verifiers with SWE-Gymcited by 336
  8. 2025WebThinker: Empowering Large Reasoning Models with Deep Research Capabilitycited by 335
  9. 2025Reinforcement Learning with Verifiable Rewards Implicitly Incentivizes Correct Reasoning in Base LLMscited by 289
  10. 2025SWE-smith: Scaling Data for Software Engineering Agentscited by 285
  11. 2025AReaL: A Large-Scale Asynchronous Reinforcement Learning System for Language Reasoningcited by 250
  12. 2025MemAgent: Reshaping Long-Context LLM with Multi-Conv RL-based Memory Agentcited by 246
  13. 2025WebSailor: Navigating Super-human Reasoning for Web Agentcited by 229
  14. 2025BrowseComp-Plus: A More Fair and Transparent Evaluation Benchmark of Deep-Research Agentcited by 200
  15. 2025Multi-SWE-bench: A Multilingual Benchmark for Issue Resolvingcited by 171
  16. 2025R2E-Gym: Procedural Environments and Hybrid Verifiers for Scaling Open-Weights SWE Agentscited by 158
  17. 2025Beyond Ten Turns: Unlocking Long-Horizon Agentic Search with Large-Scale Asynchronous RLcited by 150
  18. 2025General-Reasoner: Advancing LLM Reasoning Across All Domainscited by 135
  19. 2025LocAgent: Graph-Guided LLM Agents for Code Localizationcited by 86
  20. 2025ARPO:End-to-End Policy Optimization for GUI Agents with Experience Replaycited by 77
  21. 2025Revisiting Reinforcement Learning for LLM Reasoning from A Cross-Domain Perspectivecited by 76
  22. 2025MegaScience: Pushing the Frontiers of Post-Training Datasets for Science Reasoningcited by 59
  23. 2025Kimi-Dev: Agentless Training as Skill Prior for SWE-Agentscited by 41

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 1; 2026: 47.

48 papers.

Sort:

By area:

  1. 2025Measuring Agents in Productioncited by 37
  2. 2026SWE-World: Building Software Engineering Agents in Docker-Free Environmentscited by 25
  3. 2026SWE-Master: Unleashing the Potential of Software Engineering Agents via Post-Trainingcited by 22
  4. 2026ProRL Agent: Rollout-as-a-Service for RL Training of Multi-Turn LLM Agentscited by 20
  5. 2026WebGym: Scaling Training Environments for Visual Web Agents with Realistic Taskscited by 19
  6. 2026ECHO: Terminal Agents Learn World Models for Freecited by 19
  7. 2026On Randomness in Agentic Evalscited by 17
  8. 2026ThunderAgent: A Simple, Fast and Program-Aware Agentic Inference Systemcited by 17
  9. 2026SERA: Soft-Verified Efficient Repository Agentscited by 10
  10. 2026Open-SWE-Traces: Advancing Dual-Mode Multilingual Distillation for Software Engineering Agentscited by 10
  11. 2026TSR: Trajectory-Search Rollouts for Multi-Turn RL of LLM Agentscited by 7
  12. 2026Revisiting DAgger in the Era of LLM-Agentscited by 7
36 more
  1. 2026Diagnosing Training Inference Mismatch in LLM Reinforcement Learning via a Zero-Mismatch Referencecited by 7
  2. 2026From SWE-ZERO to SWE-HERO: Execution-free to Execution-based Fine-tuning for Software Engineering Agentscited by 6
  3. 2026From Chatbot to Digital Colleague: The Paradigm Shift Toward Persistent Autonomous AIcited by 6
  4. 2026Scaling Large Reasoning Models beyond Human Supervision: A Path toward Superintelligencecited by 5
  5. 2026boldsymbolf-OPD: Stabilizing Long-Horizon On-Policy Distillation with Freshness-Aware Controlcited by 4
  6. 2026APPO: Agentic Procedural Policy Optimizationcited by 4
  7. 2026LEGO-RL: Harness-Native Reinforcement Learning for Coding Agentscited by 4
  8. 2026ProAct: Agentic Lookahead in Interactive Environmentscited by 3
  9. 2026Rewarding Beliefs, Not Actions: Consistency-Guided Credit Assignment for Long-Horizon Agentscited by 3
  10. 2026Closing the Reflection Gap: A Free Calibration Bonus for Agentic RLcited by 3
  11. 2026ToolVerse: Unlocking Massive Environments and Long-Horizon Tasks for Agentic Reinforcement Learningcited by 3
  12. 2026ExecCritic: Learn to Test, Test to Improve for Coding Agentscited by 3
  13. 2026Rollout-Training Co-Design for Efficient LLM-Based Multi-Agent Reinforcement Learningcited by 2
  14. 2026RepoLaunch: Automating Build and Management of Code Repositories across Languages and Platformscited by 2
  15. 2026LatentGym: A Testbed For Cross-Task Experiential Learning With Controllable Latent Structurecited by 2
  16. 2026Environment Evolution for Terminal Agentscited by 2
  17. 2026DeliveryGym: An RL Environment for Long-Horizon Embodied Agent Planning with Adaptive Curriculumcited by 2
  18. 2026ORACLE-SWE: Quantifying the Contribution of Oracle Information Signals on SWE Agentscited by 1
  19. 2026Terminus-4B: Can a Smaller Model Replace Frontier LLMs at Agentic Execution Tasks?cited by 1
  20. 2026MARLaaS: Multi-Tenant Asynchronous Reinforcement Learning as a Servicecited by 1
  21. 2026AstraFlow: Dataflow-Oriented Reinforcement Learning for Agentic LLMscited by 1
  22. 2026AgentJet: A Distributed Swarm Training Framework for Agentic Reinforcement Learningcited by 1
  23. 2026A First-Principles Derivation of LLM Policy Optimization: From Expected Reward to GRPO and Its Structural Extensionscited by 1
  24. 2026The Rollout Infrastructure Tax in Coding-Agent Reinforcement Learningcited by 1
  25. 2026ARL-Tangram: Unleash the Resource Efficiency in Agentic Reinforcement Learningcited by 0
  26. 2026When to Stop Reusing: Dynamic Gradient Gating for Sample-Efficient RLVRcited by 0
  27. 2026A Matter of TASTE: Improving Coverage and Difficulty of Agent Benchmarkscited by 0
  28. 2026MintAct: A Unified Visual Agent for Digital Environmentscited by 0
  29. 2026GRRR: The Geometry of Reshaping, Rotation, and Routing in Decoder LLM post-trainingcited by 0
  30. 2026From Self-Distillation to Self-Practice: Privileged Information for Multi-Turn Agentscited by 0
  31. 2026SCLATE: a Substrate for Continual-Learning Agent Training and Evaluationcited by 0
  32. 2026EfficientAgent: What Makes KV Cache Offloading Work for Concurrent Agents?cited by 0
  33. 2026QwenGyre: An Elastic Reinforcement Learning Framework for Training xLong-Horizon Agentscited by 0
  34. 2026Counterfactual Rollout Replay: Forkable Environments as Free Process Rewards for Software Engineering Agentscited by 0
  35. 2026ANTE: Anticipate, Then Execute -- Learning Internal World Models for Coding Agents from Verified Beliefscited by 0
  36. 2026ReTerm: Recovery Training for Self-Improving Terminal Agentscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025AgentRL: Scaling Agentic Reinforcement Learning with a Multi-Turn, Multi-Task Framework
  2. 2026Agent Lightning v1.0: Towards Harnessed Agentic RL
  3. 2025Efficient Multi-turn RL for GUI Agents via Decoupled Training and Adaptive Data Curation
  4. 2025SWEET-RL: Training Multi-Turn LLM Agents on Collaborative Reasoning Tasks
  5. 2025AgentGym-RL: Training LLM Agents for Long-Horizon Decision Making through Multi-Turn Reinforcement Learning
  6. 2026Belayer: Efficient Fault Tolerance for LLM Agentic RL Training
  7. 2026Dr. MAS: Stable Reinforcement Learning for Multi-Agent LLM Systems
  8. 2026SPROUT: Scaling Agentic Reinforcement Learning for Long-Horizon Software Engineering