# A Benchmark Environment Motivated by Industrial Control Problems

arXiv: https://arxiv.org/abs/1709.09480 · HTML: https://mlsanity.com/paper/1709.09480/

## Key facts

- Cited by 31 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 27 Sep 2017.

## Built on by (most cited first, 30 of 31)

- [A Survey on Model-based Reinforcement Learning](https://mlsanity.com/paper/2206.09328/) (2022)
- [NeoRL: A Near Real-World Benchmark for Offline Reinforcement Learning](https://mlsanity.com/paper/2102.00714/) (2021)
- [Overcoming Model Bias for Robust Offline Deep Reinforcement Learning](https://mlsanity.com/paper/2008.05533/) (2020)
- [Learning Long-Term Reward Redistribution via Randomized Return Decomposition](https://mlsanity.com/paper/2111.13485/) (2021)
- [Off-Policy Reinforcement Learning with Delayed Rewards](https://mlsanity.com/paper/2106.11854/) (2021)
- [Showing Your Offline Reinforcement Learning Work: Online Evaluation Budget Matters](https://mlsanity.com/paper/2110.04156/) (2021)
- [Dynamic Datasets and Market Environments for Financial Reinforcement Learning](https://mlsanity.com/paper/2304.13174/) (2023)
- [Provably Efficient Offline Reinforcement Learning with Trajectory-Wise Reward](https://mlsanity.com/paper/2206.06426/) (2022)
- [User-Interactive Offline Reinforcement Learning](https://mlsanity.com/paper/2205.10629/) (2022)
- [Vintix: Action Model via In-Context Reinforcement Learning](https://mlsanity.com/paper/2501.19400/) (2025)
- [Comparing Model-free and Model-based Algorithms for Offline Reinforcement Learning](https://mlsanity.com/paper/2201.05433/) (2022)
- [LIFT: Reinforcement Learning in Computer Systems by Learning From Demonstrations](https://mlsanity.com/paper/1808.07903/) (2018)
- [Fully Test-time Adaptation for Tabular Data](https://mlsanity.com/paper/2412.10871/) (2024)
- [ContainerGym: A Real-World Reinforcement Learning Benchmark for Resource Allocation](https://mlsanity.com/paper/2307.02991/) (2023)
- [AdapTable: Test-Time Adaptation for Tabular Data via Shift-Aware Uncertainty Calibrator and Label Distribution Handler](https://mlsanity.com/paper/2407.10784/) (2024)
- [Behavior Constraining in Weight Space for Offline Reinforcement Learning](https://mlsanity.com/paper/2107.05479/) (2021)
- [FACTS: A Factored State-Space Framework For World Modelling](https://mlsanity.com/paper/2410.20922/) (2024)
- [Measuring Data Quality for Dataset Selection in Offline Reinforcement Learning](https://mlsanity.com/paper/2111.13461/) (2021)
- [Learning Control Policies for Variable Objectives from Offline Data](https://mlsanity.com/paper/2308.06127/) (2023)
- [SortingEnv: An Extendable RL-Environment for an Industrial Sorting Process](https://mlsanity.com/paper/2503.10466/) (2025)
- [Bellman: A Toolbox for Model-Based Reinforcement Learning in TensorFlow](https://mlsanity.com/paper/2103.14407/) (2021)
- [Episodic Return Decomposition by Difference of Implicitly Assigned Sub-Trajectory Reward](https://mlsanity.com/paper/2312.10642/) (2023)
- [Safe Deployment of Offline Reinforcement Learning via Input Convex Action Correction](https://mlsanity.com/paper/2507.22640/) (2025)
- [A cGAN Ensemble-based Uncertainty-aware Surrogate Model for Offline Model-based Optimization in Industrial Control Problems](https://mlsanity.com/paper/2205.07250/) (2022)
- [Learning Multiple Coordinated Agents under Directed Acyclic Graph Constraints](https://mlsanity.com/paper/2307.07529/) (2023)
- [Automatic Trade-off Adaptation in Offline RL](https://mlsanity.com/paper/2306.09744/) (2023)
- [Scalable Multi-agent Reinforcement Learning for Factory-wide Dynamic Scheduling](https://mlsanity.com/paper/2409.13571/) (2024)
- [Variational Quantum Circuits in Offline Contextual Bandit Problems](https://mlsanity.com/paper/2509.07633/) (2025)
- [Evolutionary Warm-Starts for Reinforcement Learning in Industrial Continuous Control](https://mlsanity.com/paper/2603.26750/) (2026)
- [Augmenting PID Control with Deep Reinforcement Learning: A Hybrid Approach to the Industrial Benchmark](https://mlsanity.com/paper/2609.22584/) (2026)

## Abstract (arXiv)

In the research area of reinforcement learning (RL), frequently novel and promising methods are developed and introduced to the RL community. However, although many researchers are keen to apply their methods on real-world problems, implementing such methods in real industry environments often is a frustrating and tedious process. Generally, academic research groups have only limited access to real industrial data and applications. For this reason, new methods are usually developed, evaluated and compared by using artificial software benchmarks. On one hand, these benchmarks are designed to provide interpretable RL training scenarios and detailed insight into the learning process of the method on hand. On the other hand, they usually do not share much similarity with industrial real-world applications. For this reason we used our industry experience to design a benchmark which bridges the gap between freely available, documented, and motivated artificial benchmarks and properties of real industrial problems. The resulting industrial benchmark (IB) has been made publicly available to the RL community by publishing its Java and Python code, including an OpenAI Gym wrapper, on Github. In this paper we motivate and describe in detail the IB's dynamics and identify prototypic experimental settings that capture common situations in real-world industry control problems.