# Safe Chance Constrained Reinforcement Learning for Batch Process Control

arXiv: https://arxiv.org/abs/2104.11706 · HTML: https://mlsanity.com/paper/2104.11706/

## Key facts

- Cited by 11 papers in the map (data through 14 Oct 2026).
- Builds on 33 of its 92 references that are in the map.
- First posted on arXiv 23 Apr 2021.

## Builds on (most cited first, 30 of 33)

- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actor](https://mlsanity.com/paper/1801.01290/) (2018)
- [Asynchronous Methods for Deep Reinforcement Learning](https://mlsanity.com/paper/1602.01783/) (2016)
- [High-Dimensional Continuous Control Using Generalized Advantage Estimation](https://mlsanity.com/paper/1506.02438/) (2015)
- [Conservative Q-Learning for Offline Reinforcement Learning](https://mlsanity.com/paper/2006.04779/) (2020)
- [Aleatoric and Epistemic Uncertainty in Machine Learning: An Introduction to Concepts and Methods](https://mlsanity.com/paper/1910.09457/) (2019)
- [A Tutorial on Bayesian Optimization](https://mlsanity.com/paper/1807.02811/) (2018)
- [MOPO: Model-based Offline Policy Optimization](https://mlsanity.com/paper/2005.13239/) (2020)
- [MOReL : Model-Based Offline Reinforcement Learning](https://mlsanity.com/paper/2005.05951/) (2020)
- [Safe Model-based Reinforcement Learning with Stability Guarantees](https://mlsanity.com/paper/1705.08551/) (2017)
- [A unified view of entropy-regularized Markov decision processes](https://mlsanity.com/paper/1705.07798/) (2017)
- [Understanding the impact of entropy on policy optimization](https://mlsanity.com/paper/1811.11214/) (2018)
- [A predictive safety filter for learning-based control of constrained nonlinear dynamical systems](https://mlsanity.com/paper/1812.05506/) (2018)
- [Efficient Model-Based Reinforcement Learning through Optimistic Policy Search and Planning](https://mlsanity.com/paper/2006.08684/) (2020)
- [Safe Reinforcement Learning Using Robust MPC](https://mlsanity.com/paper/1906.04005/) (2019)
- [A Game Theoretic Framework for Model Based Reinforcement Learning](https://mlsanity.com/paper/2004.07804/) (2020)
- [Safe Reinforcement Learning with Model Uncertainty Estimates](https://mlsanity.com/paper/1810.08700/) (2018)
- [Estimating Risk and Uncertainty in Deep Reinforcement Learning](https://mlsanity.com/paper/1905.09638/) (2019)
- [Identification of Gaussian Process State Space Models](https://mlsanity.com/paper/1705.10888/) (2017)
- [Reinforcement Learning for Batch Bioprocess Optimization](https://mlsanity.com/paper/1904.07292/) (2019)
- [Stochastic data-driven model predictive control using Gaussian processes](https://mlsanity.com/paper/1908.01786/) (2019)
- [Deep Reinforcement Learning for Process Control: A Primer for Beginners](https://mlsanity.com/paper/2004.05490/) (2020)
- [Chance Constrained Policy Optimization for Process Control and Optimization](https://mlsanity.com/paper/2008.00030/) (2020)
- [Safe Reinforcement Learning Using Robust Action Governor](https://mlsanity.com/paper/2102.10643/) (2021)
- [Safe reinforcement learning for probabilistic reachability and safety specifications: A Lyapunov-based approach](https://mlsanity.com/paper/2002.10126/) (2020)
- [Separated Proportional-Integral Lagrangian for Chance Constrained Reinforcement Learning](https://mlsanity.com/paper/2102.08539/) (2021)
- [Constrained Model-Free Reinforcement Learning for Process Optimization](https://mlsanity.com/paper/2011.07925/) (2020)
- [Receding-horizon Stochastic Model Predictive Control with Hard Input Constraints and Joint State Chance Constraints](https://mlsanity.com/paper/1506.08471/) (2015)
- [Twin actor twin delayed deep deterministic policy gradient (TATD3) learning for batch process control](https://mlsanity.com/paper/2102.13012/) (2021)
- [A Meta-Reinforcement Learning Approach to Process Control](https://mlsanity.com/paper/2103.14060/) (2021)

## Built on by (most cited first, 11 of 11)

- [A Survey of Constraint Formulations in Safe Reinforcement Learning](https://mlsanity.com/paper/2402.02025/) (2024)
- [Flowsheet synthesis through hierarchical reinforcement learning and graph neural networks](https://mlsanity.com/paper/2207.12051/) (2022)
- [PC-Gym: Benchmark Environments For Process Control Problems](https://mlsanity.com/paper/2410.22093/) (2024)
- [Anytime-Constrained Reinforcement Learning](https://mlsanity.com/paper/2311.05511/) (2023)
- [When Bioprocess Engineering Meets Machine Learning: A Survey from the Perspective of Automated Bioprocess Development](https://mlsanity.com/paper/2209.01083/) (2022)
- [Bayesian Safe Policy Learning with Chance Constrained Optimization: Application to Military Security Assessment during the Vietnam War](https://mlsanity.com/paper/2307.08840/) (2023)
- [Flipping-based Policy for Chance-Constrained Markov Decision Processes](https://mlsanity.com/paper/2410.06474/) (2024)
- [Distributional Reinforcement Learning for Scheduling of Chemical Production Processes](https://mlsanity.com/paper/2203.00636/) (2022)
- [Reinforcement Learning-based Control via Y-wise Affine Neural Networks (YANNs)](https://mlsanity.com/paper/2508.16474/) (2025)
- [Survey and Tutorial of Reinforcement Learning Methods in Process Systems Engineering](https://mlsanity.com/paper/2510.24272/) (2025)
- [Opportunities of Hybrid Model-based Reinforcement Learning for Cell Therapy Manufacturing Process Control](https://mlsanity.com/paper/2201.03116/) (2022)

## Abstract (arXiv)

Reinforcement Learning (RL) controllers have generated excitement within the control community. The primary advantage of RL controllers relative to existing methods is their ability to optimize uncertain systems independently of explicit assumption of process uncertainty. Recent focus on engineering applications has been directed towards the development of safe RL controllers. Previous works have proposed approaches to account for constraint satisfaction through constraint tightening from the domain of stochastic model predictive control. Here, we extend these approaches to account for plant-model mismatch. Specifically, we propose a data-driven approach that utilizes Gaussian processes for the offline simulation model and use the associated posterior uncertainty prediction to account for joint chance constraints and plant-model mismatch. The method is benchmarked against nonlinear model predictive control via case studies. The results demonstrate the ability of the methodology to account for process uncertainty, enabling satisfaction of joint chance constraints even in the presence of plant-model mismatch.