# SafeOR-Gym: A Benchmark Suite for Safe Reinforcement Learning Algorithms on Practical Operations Research Problems

arXiv: https://arxiv.org/abs/2506.02255 · HTML: https://mlsanity.com/paper/2506.02255/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Placed in Safe and constrained RL, one of 15 directions in Deep reinforcement learning methods.
- First posted on arXiv 2 Jun 2025.

## Placed in

- Deep reinforcement learning methods / [Safe and constrained RL](https://mlsanity.com/deeprl/safe-and-constrained-rl/)

## Built on by (most cited first, 6 of 6)

- [Survey and Tutorial of Reinforcement Learning Methods in Process Systems Engineering](https://mlsanity.com/paper/2510.24272/) (2025)
- [CRAX: Fast Safe Reinforcement Learning Benchmarking](https://mlsanity.com/paper/2606.20376/) (2026)
- [Towards a Practical Understanding of Lagrangian Methods in Safe Reinforcement Learning](https://mlsanity.com/paper/2510.17564/) (2025)
- [SkillChain-Gym: A Benchmark for Reskilling-Aware Production-Inventory Control under Disruptions](https://mlsanity.com/paper/2606.17266/) (2026)
- [Atoms to Processes: The Role of Artificial Intelligence and Machine Learning in Chemical Engineering](https://mlsanity.com/paper/2610.02014/) (2026)
- [Factoriax: A GPU-Accelerated Factorio-Style Simulator for Reinforcement Learning](https://mlsanity.com/paper/2610.05569/) (2026)

## Abstract (arXiv)

Most existing safe reinforcement learning (RL) benchmarks focus on robotics and control tasks, offering limited relevance to high-stakes domains that involve structured constraints, mixed-integer decisions, and industrial complexity. This gap hinders the advancement and deployment of safe RL in critical areas such as energy systems, manufacturing, and supply chains. To address this limitation, we present SafeOR-Gym, a benchmark suite of nine operations research (OR) environments tailored for safe RL under complex constraints. Each environment captures a realistic planning, scheduling, or control problems characterized by cost-based constraint violations, planning horizons, and hybrid discrete-continuous action spaces. The suite integrates seamlessly with the Constrained Markov Decision Process (CMDP) interface provided by OmniSafe. We evaluate several state-of-the-art safe RL algorithms across these environments, revealing a wide range of performance: while some tasks are tractable, others expose fundamental limitations in current approaches. SafeORGym provides a challenging and practical testbed that aims to catalyze future research in safe RL for real-world decision-making problems.