# Gymnasium: A Standard Interface for Reinforcement Learning Environments

arXiv: https://arxiv.org/abs/2407.17032 · HTML: https://mlsanity.com/paper/2407.17032/

## Key facts

- Cited by 46 papers in the map (data through 14 Oct 2026).
- Placed in RL for robot locomotion and manipulation, one of 15 directions in Deep reinforcement learning methods.
- Builds on 25 of its 48 references that are in the map.
- First posted on arXiv 24 Jul 2024.

## Placed in

- Deep reinforcement learning methods / [RL for robot locomotion and manipulation](https://mlsanity.com/deeprl/rl-for-robot-locomotion-and-manipulation/)

## Builds on (most cited first, 25 of 25)

- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actor](https://mlsanity.com/paper/1801.01290/) (2018)
- [Playing Atari with Deep Reinforcement Learning](https://mlsanity.com/paper/1312.5602/) (2013)
- [Meta-World: A Benchmark and Evaluation for Multi-Task and Meta Reinforcement Learning](https://mlsanity.com/paper/1910.10897/) (2019)
- [Hindsight Experience Replay](https://mlsanity.com/paper/1707.01495/) (2017)
- [Dota 2 with Large Scale Deep Reinforcement Learning](https://mlsanity.com/paper/1912.06680/) (2019)
- [Brax -- A Differentiable Physics Engine for Large Scale Rigid Body Simulation](https://mlsanity.com/paper/2106.13281/) (2021)
- [dm_control: Software and Tasks for Continuous Control](https://mlsanity.com/paper/2006.12983/) (2020)
- [Minigrid & Miniworld: Modular & Customizable Reinforcement Learning Environments for Goal-Oriented Tasks](https://mlsanity.com/paper/2306.13831/) (2023)
- [Reinforcement Learning on Web Interfaces Using Workflow-Guided Exploration](https://mlsanity.com/paper/1802.08802/) (2018)
- [PettingZoo: Gym for Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/2009.14471/) (2020)
- [ViZDoom: A Doom-based AI Research Platform for Visual Reinforcement Learning](https://mlsanity.com/paper/1605.02097/) (2016)
- [Acme: A Research Framework for Distributed Reinforcement Learning](https://mlsanity.com/paper/2006.00979/) (2020)
- [Tianshou: a Highly Modularized Deep Reinforcement Learning Library](https://mlsanity.com/paper/2107.14171/) (2021)
- [Behaviour Suite for Reinforcement Learning](https://mlsanity.com/paper/1908.03568/) (2019)
- [Dopamine: A Research Framework for Deep Reinforcement Learning](https://mlsanity.com/paper/1812.06110/) (2018)
- [EnvPool: A Highly Parallel Reinforcement Learning Environment Execution Engine](https://mlsanity.com/paper/2206.10558/) (2022)
- [Pgx: Hardware-Accelerated Parallel Game Simulators for Reinforcement Learning](https://mlsanity.com/paper/2303.17503/) (2023)
- [OCAtari: Object-Centric Atari 2600 Reinforcement Learning Environments](https://mlsanity.com/paper/2306.08649/) (2023)
- [PufferLib: Making Reinforcement Learning Libraries and Environments Play Nice](https://mlsanity.com/paper/2406.12905/) (2024)
- [Open RL Benchmark: Comprehensive Tracked Experiments for Reinforcement Learning](https://mlsanity.com/paper/2402.03046/) (2024)
- [PyFlyt -- UAV Simulation Environments for Reinforcement Learning Research](https://mlsanity.com/paper/2304.01305/) (2023)
- [Multiplayer Support for the Arcade Learning Environment](https://mlsanity.com/paper/2009.09341/) (2020)
- [ICU-Sepsis: A Benchmark MDP Built from Real Medical Data](https://mlsanity.com/paper/2406.05646/) (2024)
- [Mask Atari for Deep Reinforcement Learning as POMDP Benchmarks](https://mlsanity.com/paper/2203.16777/) (2022)

## Built on by (most cited first, 30 of 46)

- [A Survey of Reinforcement Learning for Large Reasoning Models](https://mlsanity.com/paper/2509.08827/) (2025)
- [VAGEN: Reinforcing World Model Reasoning for Multi-Turn VLM Agents](https://mlsanity.com/paper/2510.16907/) (2025)
- [Gym-Anything: Turn any Software into an Agent Environment](https://mlsanity.com/paper/2604.06126/) (2026)
- [From Code Foundation Models to Agents and Applications: A Comprehensive Survey and Practical Guide to Code Intelligence](https://mlsanity.com/paper/2511.18538/) (2025)
- [General Modular Harness for LLM Agents in Multi-Turn Gaming Environments](https://mlsanity.com/paper/2507.11633/) (2025)
- [VisGym: Diverse, Customizable, Scalable Environments for Multimodal Agents](https://mlsanity.com/paper/2601.16973/) (2026)
- [Reward Learning from Multiple Feedback Types](https://mlsanity.com/paper/2502.21038/) (2025)
- [GameVerse: Can Vision-Language Models Learn from Video-based Reflection?](https://mlsanity.com/paper/2603.06656/) (2026)
- [KORGym: A Dynamic Game Platform for LLM Reasoning Evaluation](https://mlsanity.com/paper/2505.14552/) (2025)
- [AdaReasoner: Dynamic Tool Orchestration for Iterative Visual Reasoning](https://mlsanity.com/paper/2601.18631/) (2026)
- [Learning Without Critics? Revisiting GRPO in Classical Reinforcement Learning Environments](https://mlsanity.com/paper/2511.03527/) (2025)
- [Agent-Environment Alignment via Automated Interface Generation](https://mlsanity.com/paper/2505.21055/) (2025)
- [FM-IRL: Flow-Matching for Reward Modeling and Policy Regularization in Reinforcement Learning](https://mlsanity.com/paper/2510.09222/) (2025)
- [Free Energy Risk Metrics for Systemically Safe AI: Gatekeeping Multi-Agent Study](https://mlsanity.com/paper/2502.04249/) (2025)
- [Prompted Policy Search: Reinforcement Learning through Linguistic and Numerical Reasoning in LLMs](https://mlsanity.com/paper/2511.21928/) (2025)
- [Hierarchical Experimentalist Agents](https://mlsanity.com/paper/2606.29315/) (2026)
- [GraphIR: Architecture-Level Search States for LLM-Guided Neural Architecture Evolution](https://mlsanity.com/paper/2608.01633/) (2026)
- [When Memory Lies: An Empirical Study of Spatial Memory Staleness in VLM Agents](https://mlsanity.com/paper/2608.04574/) (2026)
- [Rule-Bottleneck Reinforcement Learning: Joint Explanation and Decision Optimization for Resource Allocation with Language Agents](https://mlsanity.com/paper/2502.10732/) (2025)
- [Characterizing the Robustness of Black-Box LLM Planners Under Perturbed Observations with Adaptive Stress Testing](https://mlsanity.com/paper/2505.05665/) (2025)
- [Fine-tuning Behavioral Cloning Policies with Preference-Based Reinforcement Learning](https://mlsanity.com/paper/2509.26605/) (2025)
- [Vintix II: Decision Pre-Trained Transformer is a Scalable In-Context Reinforcement Learner](https://mlsanity.com/paper/2604.05112/) (2026)
- [What You Think is What You See: Driving Exploration in VLM Agents via Visual-Linguistic Curiosity](https://mlsanity.com/paper/2605.03782/) (2026)
- [Augmented Lagrangian Method for Last-Iterate Convergence for Constrained MDPs](https://mlsanity.com/paper/2605.11694/) (2026)
- [Double Preconditioning (DoPr): Optimization for Test-Time Performance, not Validation Loss](https://mlsanity.com/paper/2606.06418/) (2026)
- [World Models Under Asynchronous Sensor Observations](https://mlsanity.com/paper/2609.07299/) (2026)
- [Putting the Iterative Training of Decision Trees to the Test on a Real-World Robotic Task](https://mlsanity.com/paper/2412.04974/) (2024)
- [HRM-Agent: Training a recurrent reasoning model in dynamic environments using reinforcement learning](https://mlsanity.com/paper/2510.22832/) (2025)
- [Operator Models for Continuous-Time Offline Reinforcement Learning](https://mlsanity.com/paper/2511.10383/) (2025)
- [Online World Modeling Enables Real-World Inverse Reinforcement Learning from Observation](https://mlsanity.com/paper/2602.24121/) (2026)

## Abstract (arXiv)

Reinforcement Learning (RL) is a continuously growing field that has the potential to revolutionize many areas of artificial intelligence. However, despite its promise, RL research is often hindered by the lack of standardization in environment and algorithm implementations. This makes it difficult for researchers to compare and build upon each other's work, slowing down progress in the field. Gymnasium is an open-source library that provides a standard API for RL environments, aiming to tackle this issue. Gymnasium's main feature is a set of abstractions that allow for wide interoperability between environments and training algorithms, making it easier for researchers to develop and test RL algorithms. In addition, Gymnasium provides a collection of easy-to-use environments, tools for easily customizing environments, and tools to ensure the reproducibility and robustness of RL research. Through this unified framework, Gymnasium significantly streamlines the process of developing and testing RL algorithms, enabling researchers to focus more on innovation and less on implementation details. By providing a standardized platform for RL research, Gymnasium helps to drive forward the field of reinforcement learning and unlock its full potential. Gymnasium is available online at https://github.com/Farama-Foundation/Gymnasium