# Gym4ReaL: A Suite for Benchmarking Real-World Reinforcement Learning

arXiv: https://arxiv.org/abs/2507.00257 · HTML: https://mlsanity.com/paper/2507.00257/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Placed in Deep RL algorithms and representations, one of 15 directions in Deep reinforcement learning methods.
- Builds on 8 of its 32 references that are in the map.
- First posted on arXiv 30 Jun 2025.

## Placed in

- Deep reinforcement learning methods / [Deep RL algorithms and representations](https://mlsanity.com/deeprl/deep-rl-algorithms-and-representations/)

## Builds on (most cited first, 8 of 8)

- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [Mastering Chess and Shogi by Self-Play with a General Reinforcement Learning Algorithm](https://mlsanity.com/paper/1712.01815/) (2017)
- [MetaDrive: Composing Diverse Driving Scenarios for Generalizable Reinforcement Learning](https://mlsanity.com/paper/2109.12674/) (2021)
- [An empirical investigation of the challenges of real-world reinforcement learning](https://mlsanity.com/paper/2003.11881/) (2020)
- [POPGym: Benchmarking Partially Observable Reinforcement Learning](https://mlsanity.com/paper/2303.01859/) (2023)
- [FinRL: Deep Reinforcement Learning Framework to Automate Trading in Quantitative Finance](https://mlsanity.com/paper/2111.09395/) (2021)
- [safe-control-gym: a Unified Benchmark Suite for Safe Learning-based Control and Reinforcement Learning in Robotics](https://mlsanity.com/paper/2109.06325/) (2021)
- [skrl: Modular and Flexible Library for Reinforcement Learning](https://mlsanity.com/paper/2202.03825/) (2022)

## Built on by (most cited first, 3 of 3)

- [Tracking Drift: Variation-Aware Entropy Scheduling for Non-Stationary Reinforcement Learning](https://mlsanity.com/paper/2601.19624/) (2026)
- [Learning in Markov Decision Processes with Exogenous Dynamics](https://mlsanity.com/paper/2603.02862/) (2026)
- [Breaking the Epistemic Trap: Active Perception Under Compound Uncertainty](https://mlsanity.com/paper/2605.26627/) (2026)

## Abstract (arXiv)

In recent years, Reinforcement Learning (RL) has made remarkable progress, achieving superhuman performance in a wide range of simulated environments. As research moves toward deploying RL in real-world applications, the field faces a new set of challenges inherent to real-world settings, such as large state-action spaces, non-stationarity, and partial observability. Despite their importance, these challenges are often underexplored in current benchmarks, which tend to focus on idealized, fully observable, and stationary environments, often neglecting to incorporate real-world complexities explicitly. In this paper, we introduce textttGym4ReaL, a comprehensive suite of realistic environments designed to support the development and evaluation of RL algorithms that can operate in real-world scenarios. The suite includes a diverse set of tasks that expose algorithms to a variety of practical challenges. Our experimental results show that, in these settings, standard RL algorithms confirm their competitiveness against rule-based benchmarks, motivating the development of new methods to fully exploit the potential of RL to tackle the complexities of real-world tasks.