mlsanityData to 7 Oct 2026

ARLArena: A Unified Framework for Stable Agentic Reinforcement Learning

arXiv 2602.21534 ↗First posted 25 Feb 2026cs.AIcited by 11

On the map: LLM agents and tool use → Reinforcement learning for LLM agents

Abstract (arXiv)

Agentic reinforcement learning (ARL) has rapidly gained attention as a promising paradigm for training agents to solve complex, multi-step interactive tasks. Despite encouraging early results, ARL remains highly unstable, often leading to training collapse. This instability limits scalability to larger environments and longer interaction horizons, and constrains systematic exploration of algorithmic design choices. In this paper, we first propose ARLArena, a stable training recipe and systematic analysis framework that examines training stability in a controlled and reproducible setting. ARLArena first constructs a clean and standardized testbed. Then, we decompose policy gradient into four core design dimensions and assess the performance and stability of each dimension. Through this fine-grained analysis, we distill a unified perspective on ARL and propose SAMPO, a stable agentic policy optimization method designed to mitigate the dominant sources of instability in ARL. Empirically, SAMPO achieves consistently stable training and strong performance across diverse agentic tasks. Overall, this study provides a unifying policy gradient perspective for ARL and offers practical guidance for building stable and reproducible LLM-based agent training pipelines.

Key facts. Cited by 11 papers in the map (data through 7 Oct 2026). Placed in Reinforcement learning for LLM agents, one of 16 directions in LLM agents and tool use. Builds on 35 of its 63 references that are in the map. First posted on arXiv 25 Feb 2026.

Builds on

35 of 63 references.

Closest ancestors · 13

Same direction, newest first

  1. 2026Latent Poincar'e Shaping for Agentic Reinforcement Learning10 Feb
  2. 2026CLEANER: Self-Purified Trajectories Boost Agentic Reinforcement Learning21 Jan
  3. 2025Demystifying Reinforcement Learning in Agentic Reasoning13 Oct
  4. 2025AgentRL: Scaling Agentic Reinforcement Learning with a Multi-Turn, Multi-Task Framework5 Oct
  5. 2025Harnessing Uncertainty: Entropy-Modulated Policy Gradients for Long-Horizon LLM Agents11 Sep
  6. 2025AgentGym-RL: Training LLM Agents for Long-Horizon Decision Making through Multi-Turn Reinforcement Learning10 Sep
  7. 2025SimpleTIR: End-to-End Reinforcement Learning for Multi-Turn Tool-Integrated Reasoning2 Sep
  8. 2025VerlTool: Towards Holistic Agentic Reinforcement Learning with Tool Use1 Sep
  9. 2025Group-in-Group Policy Optimization for LLM Agent Training16 May
  10. 2025RAGEN: Understanding Self-Evolution in LLM Agents via Multi-Turn Reinforcement Learning24 Apr
  11. 2025SWEET-RL: Training Multi-Turn LLM Agents on Collaborative Reasoning Tasks19 Mar
  12. 2024AGILE: A Novel Reinforcement Learning Framework of LLM Agents23 May
1 more
  1. 2023Voyager: An Open-Ended Embodied Agent with Large Language Models25 May

Other references · 22

  1. 2022Training language models to follow instructions with human feedbackcited by 15,712
  2. 2017Proximal Policy Optimization Algorithmscited by 14,441
  3. 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,355
  4. 2025Qwen3 Technical Reportcited by 8,195
  5. 2025DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learningcited by 6,146
  6. 2023Toolformer: Language Models Can Teach Themselves to Use Toolscited by 4,072
  7. 2025DAPO: An Open-Source LLM Reinforcement Learning System at Scalecited by 2,678
  8. 2024HybridFlow: A Flexible and Efficient RLHF Frameworkcited by 1,980
  9. 2025Search-R1: Training LLMs to Reason and Leverage Search Engines with Reinforcement Learningcited by 1,371
  10. 2022WebShop: Towards Scalable Real-World Web Interaction with Grounded Language Agentscited by 1,197
  11. 2020ALFWorld: Aligning Text and Embodied Environments for Interactive Learningcited by 1,143
  12. 2025Group Sequence Policy Optimizationcited by 788
10 more
  1. 2025DeepSeek-V3.2: Pushing the Frontier of Open Large Language Modelscited by 755
  2. 2023API-Bank: A Comprehensive Benchmark for Tool-Augmented LLMscited by 574
  3. 2025SimpleRL-Zoo: Investigating and Taming Zero Reinforcement Learning for Open Base Models in the Wildcited by 505
  4. 2025MiniMax-M1: Scaling Test-Time Compute Efficiently with Lightning Attentioncited by 248
  5. 2025Agentic Large Language Models, a surveycited by 105
  6. 2025The Art of Scaling Reinforcement Learning Compute for LLMscited by 91
  7. 2025Soft Adaptive Policy Optimizationcited by 89
  8. 2023LMRL Gym: Benchmarks for Multi-Turn Reinforcement Learning with Language Modelscited by 84
  9. 2026A Survey of Agentic Reasoning for Large Language Models: Towards Recursively Self-Improving and Collective Agentscited by 47
  10. 2026AgentArk: Distilling Multi-Agent Intelligence into a Single LLM Agentcited by 12

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 11.

11 papers.

  1. 2026Agentic Environment Engineering for Large Language Models: A Survey of Environment Modeling, Synthesis, Evaluation, and Applicationcited by 8
  2. 2026LongCat-Flash-Prover: Advancing Native Formal Reasoning via Agentic Tool-Integrated Reinforcement Learningcited by 6
  3. 2026Resolving Action Bottleneck: Agentic Reinforcement Learning Informed by Token-Level Energycited by 5
  4. 2026Training AI Scientists to Replicate Researchcited by 4
  5. 2026Improving Search Agent with One Line of Codecited by 2
  6. 2026Co-Evolution of Policy and Internal Reward for Language Agentscited by 1
  7. 2026T2PO: Uncertainty-Guided Exploration Control for Stable Multi-Turn Agentic Reinforcement Learningcited by 1
  8. 2026Retrieval, Reward, and Training Protocols: What Matters in Training Search Agents?cited by 1
  9. 2026HarnessBridge: Learnable Bidirectional Controller for LLM Agent Harnesscited by 1
  10. 2026ACLArena: Agent Continue Learning in Multi-stage Post-trainingcited by 0
  11. 2026Semifactual Credit-Augmented Policy Optimizationcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025The Landscape of Agentic Reinforcement Learning for LLMs: A Survey
  2. 2025Agentic Reinforced Policy Optimization
  3. 2026Molt: A Scalable PyTorch-Native Training Framework for Agentic Reinforcement Learning
  4. 2026AgentJet: A Distributed Swarm Training Framework for Agentic Reinforcement Learning
  5. 2026Agentic Monte Carlo: Simulating Reinforcement Learning for Black-Box Agents
  6. 2025Agent-R1: A Unified and Modular Framework for Agentic Reinforcement Learning
  7. 2026RTPO: Reverse-Turn Policy Optimization for Stabilizing Agentic RL Training
  8. 2025AutoForge: Automated Environment Synthesis for Agentic Reinforcement Learning