# ContainerGym: A Real-World Reinforcement Learning Benchmark for Resource Allocation

arXiv: https://arxiv.org/abs/2307.02991 · HTML: https://mlsanity.com/paper/2307.02991/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Builds on 8 of its 17 references that are in the map.
- First posted on arXiv 6 Jul 2023.

## Builds on (most cited first, 8 of 8)

- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [Playing Atari with Deep Reinforcement Learning](https://mlsanity.com/paper/1312.5602/) (2013)
- [Trust Region Policy Optimization](https://mlsanity.com/paper/1502.05477/) (2015)
- [OpenAI Gym](https://mlsanity.com/paper/1606.01540/) (2016)
- [Soft Actor-Critic Algorithms and Applications](https://mlsanity.com/paper/1812.05905/) (2018)
- [An empirical investigation of the challenges of real-world reinforcement learning](https://mlsanity.com/paper/2003.11881/) (2020)
- [Simulation-based reinforcement learning for real-world autonomous driving](https://mlsanity.com/paper/1911.12905/) (2019)
- [A Benchmark Environment Motivated by Industrial Control Problems](https://mlsanity.com/paper/1709.09480/) (2017)

## Built on by (most cited first, 6 of 6)

- [Multi-Agent Reinforcement Learning for Resources Allocation Optimization: A Survey](https://mlsanity.com/paper/2504.21048/) (2025)
- [SortingEnv: An Extendable RL-Environment for an Industrial Sorting Process](https://mlsanity.com/paper/2503.10466/) (2025)
- [Leveraging Genetic Algorithms for Efficient Demonstration Generation in Real-World Reinforcement Learning Environments](https://mlsanity.com/paper/2507.00762/) (2025)
- [Solving a Real-World Optimization Problem Using Proximal Policy Optimization with Curriculum Learning and Reward Engineering](https://mlsanity.com/paper/2404.02577/) (2024)
- [Curriculum RL meets Monte Carlo Planning: Optimization of a Real World Container Management Problem](https://mlsanity.com/paper/2503.17194/) (2025)
- [Balancing Specialization and Centralization: A Multi-Agent Reinforcement Learning Benchmark for Sequential Industrial Control](https://mlsanity.com/paper/2510.20408/) (2025)

## Abstract (arXiv)

We present ContainerGym, a benchmark for reinforcement learning inspired by a real-world industrial resource allocation task. The proposed benchmark encodes a range of challenges commonly encountered in real-world sequential decision making problems, such as uncertainty. It can be configured to instantiate problems of varying degrees of difficulty, e.g., in terms of variable dimensionality. Our benchmark differs from other reinforcement learning benchmarks, including the ones aiming to encode real-world difficulties, in that it is directly derived from a real-world industrial problem, which underwent minimal simplification and streamlining. It is sufficiently versatile to evaluate reinforcement learning algorithms on any real-world problem that fits our resource allocation framework. We provide results of standard baseline methods. Going beyond the usual training reward curves, our results and the statistical tools used to interpret them allow to highlight interesting limitations of well-known deep reinforcement learning algorithms, namely PPO, TRPO and DQN.