mlsanityData to 14 Oct 2026

GEM: A Gym for Agentic LLMs

arXiv 2510.01051 ↗First posted 1 Oct 2025cs.LG cs.AI cs.CLcited by 23

On the map: LLM agents and tool use → Reinforcement learning for LLM agents

Abstract (arXiv)

The training paradigm for large language models (LLMs) is moving from static datasets to experience-based learning, where agents acquire skills via interacting with complex environments. To facilitate this transition we introduce GEM (General Experience Maker), an open-source environment simulator designed for the age of LLMs. Analogous to OpenAI-Gym for traditional reinforcement learning (RL), GEM provides a standardized framework for the environment-agent interface, including asynchronous vectorized execution for high throughput, and flexible wrappers for easy extensibility. GEM also features a diverse suite of environments, robust integrated tools, and single-file example scripts demonstrating using GEM with five popular RL training frameworks. Along with this, we also provide a set of baselines across 24 environments using REINFORCE with Return Batch Normalization (ReBN), which -- unlike GRPO -- is compatible with the full RL setting of dense per-turn rewards and offers better credit assignment. We further conduct apple-to-apple benchmarking of PPO, GRPO and REINFORCE in both single- and multi-turn settings using GEM to shed light on the algorithmic designs. Lastly, GEM also functions as a convenient evaluation toolkit besides a training environment. We hope this framework can help accelerate future agentic LLM research.

Key facts. Cited by 23 papers in the map (data through 14 Oct 2026). Placed in Reinforcement learning for LLM agents, one of 16 directions in LLM agents and tool use. Builds on 29 of its 56 references that are in the map. First posted on arXiv 1 Oct 2025.

Builds on

29 of 56 references.

Closest ancestors · 4

Same direction, newest first

  1. 2025SimpleTIR: End-to-End Reinforcement Learning for Multi-Turn Tool-Integrated Reasoning2 Sep
  2. 2025SPIRAL: Self-Play on Zero-Sum Games Incentivizes Reasoning via Multi-Agent Multi-Turn Reinforcement Learning30 Jun
  3. 2025Group-in-Group Policy Optimization for LLM Agent Training16 May
  4. 2025ReTool: Reinforcement Learning for Strategic Tool Use in LLMs15 Apr

Other references · 25

  1. 2022Training language models to follow instructions with human feedbackcited by 15,971
  2. 2017Proximal Policy Optimization Algorithmscited by 15,215
  3. 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 9,379
  4. 2025Qwen3 Technical Reportcited by 8,820
  5. 2023Efficient Memory Management for Large Language Model Serving with PagedAttentioncited by 5,944
  6. 2025Qwen2.5-VL Technical Reportcited by 5,587
  7. 2019Fine-Tuning Language Models from Human Preferencescited by 2,150
  8. 2024HybridFlow: A Flexible and Efficient RLHF Frameworkcited by 2,057
  9. 2015High-Dimensional Continuous Control Using Generalized Advantage Estimationcited by 2,026
  10. 2025Search-R1: Training LLMs to Reason and Leverage Search Engines with Reinforcement Learningcited by 1,444
  11. 2024τ-bench: A Benchmark for Tool-Agent-User Interaction in Real-World Domainscited by 1,402
  12. 2025Understanding R1-Zero-Like Training: A Critical Perspectivecited by 1,345
13 more
  1. 2024Tulu 3: Pushing Frontiers in Open Language Model Post-Trainingcited by 1,053
  2. 2023Beyond Human Data: Scaling Self-Training for Problem-Solving with Language Modelscited by 349
  3. 2019OpenSpiel: A Framework for Reinforcement Learning in Gamescited by 186
  4. 2024VinePPO: Refining Credit Assignment in RL Training of LLMscited by 183
  5. 2018Deep Reinforcement Learning and the Deadly Triadcited by 174
  6. 2025ReSearch: Learning to Reason with Search for LLMs via Reinforcement Learningcited by 161
  7. 2025A Minimalist Approach to LLM Reasoning: from Rejection Sampling to Reinforcecited by 145
  8. 2025REASONING GYM: Reasoning Environments for Reinforcement Learning with Verifiable Rewardscited by 130
  9. 2025Reinforcement Learning Optimization for Large-Scale Learning: An Efficient and User-Friendly Scaling Librarycited by 101
  10. 2023LMRL Gym: Benchmarks for Multi-Turn Reinforcement Learning with Language Modelscited by 88
  11. 2023TorchRL: A data-driven decision-making library for PyTorchcited by 60
  12. 2022MARLlib: A Scalable and Efficient Multi-agent Reinforcement Learning Librarycited by 21
  13. 2024Open RL Benchmark: Comprehensive Tracked Experiments for Reinforcement Learningcited by 17

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 3; 2026: 20.

23 papers.

Sort:

By area:

  1. 2025SkyRL-Agent: Efficient RL Training for Multi-turn LLM Agentcited by 48
  2. 2026Rethinking the Trust Region in LLM Reinforcement Learningcited by 37
  3. 2026ProRL Agent: Rollout-as-a-Service for RL Training of Multi-Turn LLM Agentscited by 20
  4. 2025Scaling Agentic Reinforcement Learning for Tool-Integrated Reasoning in VLMscited by 17
  5. 2026RetroAgent: From Solving to Evolving via Retrospective Dual Intrinsic Feedbackcited by 17
  6. 2026LOCA-bench: Benchmarking Language Agents Under Controllable and Extreme Context Growthcited by 14
  7. 2025Environment Scaling for Interactive Agentic Experience Collection: A Surveycited by 13
  8. 2026SPADE: Self-Play in Adaptive Synthetic Executable Environmentscited by 12
  9. 2026Verifiable Process Rewards for Agentic Reasoningcited by 11
  10. 2026Agentic Environment Engineering for Large Language Models: A Survey of Environment Modeling, Synthesis, Evaluation, and Applicationcited by 8
  11. 2026On Training Large Language Models for Long-Horizon Tasks: An Empirical Study of Horizon Lengthcited by 6
  12. 2026Gym-V: A Unified Vision Environment System for Agentic Vision Researchcited by 3
11 more
  1. 2026Agentic ESOpt: Fine-Tuning Long-Horizon LLM Agents with Minimal GPU Requirementscited by 3
  2. 2026LatentGym: A Testbed For Cross-Task Experiential Learning With Controllable Latent Structurecited by 2
  3. 2026CASCADE: Case-Based Continual Adaptation for Large Language Models During Deploymentcited by 1
  4. 2026Harbor Adapters and Harbor-Index: Infrastructure and a Curated Meta-Dataset for Large-Scale Agentic Evaluationcited by 1
  5. 2026Towards Autonomous Business Intelligence via Data-to-Insight Discovery Agentcited by 0
  6. 2026Environments as Scaffold: Enriching Feedback to Bootstrap Self-Evolving Agents in Long-Horizon Taskscited by 0
  7. 2026ProCredit: From Outcome Rewards to Progress Credit in Agentic Reinforcement Learningcited by 0
  8. 2026NeutronGym: Physics-Graded Neutron Instrument Design for LLM Agentscited by 0
  9. 2026InvestigationWorlds: An Agentic Environment for Legal Investigationcited by 0
  10. 2026BACAM: Behavior-Aware Continual Agent Merging for Multi-Turn Interactioncited by 0
  11. 2026Sherpa: Teaching LLMs to Teach Adaptivelycited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026Mock Worlds, Real Skills: Building Small Agentic Language Models with Synthetic Tasks, Simulated Environments, and Rubric-Based Rewards
  2. 2026Connect the Dots: Training LLMs for Long-Lifecycle Agents with Cross-Domain Generalization Via Reinforcement Learning
  3. 2024EnvGen: Generating and Adapting Environments via LLMs for Training Embodied Agents
  4. 2025ML-Agent: Reinforcing LLM Agents for Autonomous Machine Learning Engineering
  5. 2024A Survey On Enhancing Reinforcement Learning in Complex Environments: Insights from Human and LLM Feedback
  6. 2026From Trainee to Trainer: LLM-Designed Training Environment for RL with Multi-Agent Reasoning
  7. 2025GenEnv: Difficulty-Aligned Co-Evolution Between LLM Agents and Environment Simulators
  8. 2024Online Intrinsic Rewards for Decision Making Agents from Large Language Model Feedback