# PC-Gym: Benchmark Environments For Process Control Problems

arXiv: https://arxiv.org/abs/2410.22093 · HTML: https://mlsanity.com/paper/2410.22093/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- Placed in RL for energy, networks and operations, one of 15 directions in Deep reinforcement learning methods.
- Builds on 19 of its 51 references that are in the map.
- First posted on arXiv 29 Oct 2024.

## Placed in

- Deep reinforcement learning methods / [RL for energy, networks and operations](https://mlsanity.com/deeprl/rl-for-energy-networks-and-operations/)

## Builds on (most cited first, 19 of 19)

- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actor](https://mlsanity.com/paper/1801.01290/) (2018)
- [Continuous control with deep reinforcement learning](https://mlsanity.com/paper/1509.02971/) (2015)
- [Constrained Policy Optimization](https://mlsanity.com/paper/1705.10528/) (2017)
- [Brax -- A Differentiable Physics Engine for Large Scale Rigid Body Simulation](https://mlsanity.com/paper/2106.13281/) (2021)
- [Understanding the impact of entropy on policy optimization](https://mlsanity.com/paper/1811.11214/) (2018)
- [Jumanji: a Diverse Suite of Scalable Reinforcement Learning Environments in JAX](https://mlsanity.com/paper/2306.09884/) (2023)
- [OR-Gym: A Reinforcement Learning Library for Operations Research Problems](https://mlsanity.com/paper/2008.06319/) (2020)
- [Pgx: Hardware-Accelerated Parallel Game Simulators for Reinforcement Learning](https://mlsanity.com/paper/2303.17503/) (2023)
- [Deep Reinforcement Learning with Shallow Controllers: An Experimental Application to PID Tuning](https://mlsanity.com/paper/2111.07171/) (2021)
- [Chance Constrained Policy Optimization for Process Control and Optimization](https://mlsanity.com/paper/2008.00030/) (2020)
- [Separated Proportional-Integral Lagrangian for Chance Constrained Reinforcement Learning](https://mlsanity.com/paper/2102.08539/) (2021)
- [Meta-Reinforcement Learning for the Tuning of PI Controllers: An Offline Approach](https://mlsanity.com/paper/2203.09661/) (2022)
- [Safe Chance Constrained Reinforcement Learning for Batch Process Control](https://mlsanity.com/paper/2104.11706/) (2021)
- [Constrained Model-Free Reinforcement Learning for Process Optimization](https://mlsanity.com/paper/2011.07925/) (2020)
- [Control-Informed Reinforcement Learning for Chemical Processes](https://mlsanity.com/paper/2408.13566/) (2024)
- [Receding-horizon Stochastic Model Predictive Control with Hard Input Constraints and Joint State Chance Constraints](https://mlsanity.com/paper/1506.08471/) (2015)
- [DiffLoop: Tuning PID controllers by differentiating through the feedback loop](https://mlsanity.com/paper/2106.10516/) (2021)
- [Beyond Expected Return: Accounting for Policy Reproducibility when Evaluating Reinforcement Learning Algorithms](https://mlsanity.com/paper/2312.07178/) (2023)

## Built on by (most cited first, 10 of 10)

- [TalkToAgent: A Human-centric Explanation of Reinforcement Learning Agents with Large Language Models](https://mlsanity.com/paper/2509.04809/) (2025)
- [Safe Deployment of Offline Reinforcement Learning via Input Convex Action Correction](https://mlsanity.com/paper/2507.22640/) (2025)
- [Survey and Tutorial of Reinforcement Learning Methods in Process Systems Engineering](https://mlsanity.com/paper/2510.24272/) (2025)
- [Local-Global Learning of Interpretable Control Policies: The Interface between MPC and Reinforcement Learning](https://mlsanity.com/paper/2503.13289/) (2025)
- [To Learn or Not to Learn: A Litmus Test for Using Reinforcement Learning in Control](https://mlsanity.com/paper/2604.11463/) (2026)
- [When (and How) to Trust the Expert: Diagnosing Query-Time Expert-Guided Reinforcement Learning](https://mlsanity.com/paper/2605.09109/) (2026)
- [Reinforcement Learning-based Control via Y-wise Affine Neural Networks: Comparative Case Studies for Chemical Processes](https://mlsanity.com/paper/2605.21211/) (2026)
- [Numbers Beat Words: A Rigorous On-Premise Benchmark for Coupled MIMO Controller Tuning](https://mlsanity.com/paper/2606.11015/) (2026)
- [ChemWorld: Programmable Chemical Worlds for Controlled and Replayable Agent Experimentation](https://mlsanity.com/paper/2608.10792/) (2026)
- [Atoms to Processes: The Role of Artificial Intelligence and Machine Learning in Chemical Engineering](https://mlsanity.com/paper/2610.02014/) (2026)

## Abstract (arXiv)

PC-Gym is an open-source tool for developing and evaluating reinforcement learning (RL) algorithms in chemical process control. It features environments that simulate various chemical processes, incorporating nonlinear dynamics, disturbances, and constraints. The tool includes customizable constraint handling, disturbance generation, reward function design, and enables comparison of RL algorithms against Nonlinear Model Predictive Control (NMPC) across different scenarios. Case studies demonstrate the framework's effectiveness in evaluating RL approaches for systems like continuously stirred tank reactors, multistage extraction processes, and crystallization reactors. The results reveal performance gaps between RL algorithms and NMPC oracles, highlighting areas for improvement and enabling benchmarking. By providing a standardized platform, PC-Gym aims to accelerate research at the intersection of machine learning, control, and process systems engineering. By connecting theoretical RL advances with practical industrial process control applications, offering researchers a tool for exploring data-driven control solutions.