# OR-Gym: A Reinforcement Learning Library for Operations Research Problems

arXiv: https://arxiv.org/abs/2008.06319 · HTML: https://mlsanity.com/paper/2008.06319/

## Key facts

- Cited by 51 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 14 Aug 2020.

## Built on by (most cited first, 30 of 51)

- [Combinatorial optimization and reasoning with graph neural networks](https://mlsanity.com/paper/2102.09544/) (2021)
- [Jumanji: a Diverse Suite of Scalable Reinforcement Learning Environments in JAX](https://mlsanity.com/paper/2306.09884/) (2023)
- [A Survey for Solving Mixed Integer Programming via Machine Learning](https://mlsanity.com/paper/2203.02878/) (2022)
- [Hindsight Learning for MDPs with Exogenous Inputs](https://mlsanity.com/paper/2207.06272/) (2022)
- [InvAgent: A Large Language Model based Multi-Agent System for Inventory Management in Supply Chains](https://mlsanity.com/paper/2407.11384/) (2024)
- [Energy-Guided Diffusion Sampling for Offline-to-Online Reinforcement Learning](https://mlsanity.com/paper/2407.12448/) (2024)
- [Distributional MIPLIB: a Multi-Domain Library for Advancing ML-Guided MILP Methods](https://mlsanity.com/paper/2406.06954/) (2024)
- [Adaptive Discretization in Online Reinforcement Learning](https://mlsanity.com/paper/2110.15843/) (2021)
- [D5RL: Diverse Datasets for Data-Driven Deep Reinforcement Learning](https://mlsanity.com/paper/2408.08441/) (2024)
- [An Analysis of Multi-Agent Reinforcement Learning for Decentralized Inventory Control Systems](https://mlsanity.com/paper/2307.11432/) (2023)
- [A Versatile Multi-Agent Reinforcement Learning Benchmark for Inventory Management](https://mlsanity.com/paper/2306.07542/) (2023)
- [PC-Gym: Benchmark Environments For Process Control Problems](https://mlsanity.com/paper/2410.22093/) (2024)
- [Value Enhancement of Reinforcement Learning via Efficient and Robust Trust Region Optimization](https://mlsanity.com/paper/2301.02220/) (2023)
- [Neural Stochastic Dual Dynamic Programming](https://mlsanity.com/paper/2112.00874/) (2021)
- [Deep Policy Iteration with Integer Programming for Inventory Management](https://mlsanity.com/paper/2112.02215/) (2021)
- [Towards Automatic Actor-Critic Solutions to Continuous Control](https://mlsanity.com/paper/2106.08918/) (2021)
- [Multi-Agent Reinforcement Learning with Shared Resources for Inventory Management](https://mlsanity.com/paper/2212.07684/) (2022)
- [XuanCe: A Comprehensive and Unified Deep Reinforcement Learning Library](https://mlsanity.com/paper/2312.16248/) (2023)
- [Interpretable Reinforcement Learning via Neural Additive Models for Inventory Management](https://mlsanity.com/paper/2303.10382/) (2023)
- [RLOR: A Flexible Framework of Deep Reinforcement Learning for Operation Research](https://mlsanity.com/paper/2303.13117/) (2023)
- [A reinforcement learning approach to resource allocation in genomic selection](https://mlsanity.com/paper/2107.10901/) (2021)
- [Comparing Deep Reinforcement Learning Algorithms in Two-Echelon Supply Chains](https://mlsanity.com/paper/2204.09603/) (2022)
- [QGym: Scalable Simulation and Benchmarking of Queuing Network Controllers](https://mlsanity.com/paper/2410.06170/) (2024)
- [ICNN-enhanced 2SP: Leveraging input convex neural networks for solving two-stage stochastic programming](https://mlsanity.com/paper/2505.05261/) (2025)
- [Towards Standardising Reinforcement Learning Approaches for Production Scheduling Problems](https://mlsanity.com/paper/2104.08196/) (2021)
- [Distributional Reinforcement Learning for Scheduling of Chemical Production Processes](https://mlsanity.com/paper/2203.00636/) (2022)
- [Distributional constrained reinforcement learning for supply chain optimization](https://mlsanity.com/paper/2302.01727/) (2023)
- [Cooperative Multi-Agent Reinforcement Learning for Inventory Management](https://mlsanity.com/paper/2304.08769/) (2023)
- [Survey and Tutorial of Reinforcement Learning Methods in Process Systems Engineering](https://mlsanity.com/paper/2510.24272/) (2025)
- [Hyperparameter Selection Methods for Fitted Q-Evaluation with Error Guarantee](https://mlsanity.com/paper/2201.02300/) (2022)

## Abstract (arXiv)

Reinforcement learning (RL) has been widely applied to game-playing and surpassed the best human-level performance in many domains, yet there are few use-cases in industrial or commercial settings. We introduce OR-Gym, an open-source library for developing reinforcement learning algorithms to address operations research problems. In this paper, we apply reinforcement learning to the knapsack, multi-dimensional bin packing, multi-echelon supply chain, and multi-period asset allocation model problems, as well as benchmark the RL solutions against MILP and heuristic models. These problems are used in logistics, finance, engineering, and are common in many business operation settings. We develop environments based on prototypical models in the literature and implement various optimization and heuristic models in order to benchmark the RL results. By re-framing a series of classic optimization problems as RL tasks, we seek to provide a new tool for the operations research community, while also opening those in the RL community to many of the problems and challenges in the OR field.