# Constrained Model-Free Reinforcement Learning for Process Optimization

arXiv: https://arxiv.org/abs/2011.07925 · HTML: https://mlsanity.com/paper/2011.07925/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 16 Nov 2020.

## Built on by (most cited first, 9 of 9)

- [Deep Reinforcement Learning for Inverse Inorganic Materials Design](https://mlsanity.com/paper/2210.11931/) (2022)
- [Safe Chance Constrained Reinforcement Learning for Batch Process Control](https://mlsanity.com/paper/2104.11706/) (2021)
- [PC-Gym: Benchmark Environments For Process Control Problems](https://mlsanity.com/paper/2410.22093/) (2024)
- [When Bioprocess Engineering Meets Machine Learning: A Survey from the Perspective of Automated Bioprocess Development](https://mlsanity.com/paper/2209.01083/) (2022)
- [Distributional Reinforcement Learning for Scheduling of Chemical Production Processes](https://mlsanity.com/paper/2203.00636/) (2022)
- [Learning Cooperative Oversubscription for Cloud by Chance-Constrained Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/2211.11759/) (2022)
- [Survey and Tutorial of Reinforcement Learning Methods in Process Systems Engineering](https://mlsanity.com/paper/2510.24272/) (2025)
- [Opportunities of Hybrid Model-based Reinforcement Learning for Cell Therapy Manufacturing Process Control](https://mlsanity.com/paper/2201.03116/) (2022)
- [Addressing Terminal Constraints in Data-Driven Demand Response Scheduling](https://mlsanity.com/paper/2605.14741/) (2026)

## Abstract (arXiv)

Reinforcement learning (RL) is a control approach that can handle nonlinear stochastic optimal control problems. However, despite the promise exhibited, RL has yet to see marked translation to industrial practice primarily due to its inability to satisfy state constraints. In this work we aim to address this challenge. We propose an 'oracle'-assisted constrained Q-learning algorithm that guarantees the satisfaction of joint chance constraints with a high probability, which is crucial for safety critical tasks. To achieve this, constraint tightening (backoffs) are introduced and adjusted using Broyden's method, hence making them self-tuned. This results in a general methodology that can be imbued into approximate dynamic programming-based algorithms to ensure constraint satisfaction with high probability. Finally, we present case studies that analyze the performance of the proposed approach and compare this algorithm with model predictive control (MPC). The favorable performance of this algorithm signifies a step toward the incorporation of RL into real world optimization and control of engineering systems, where constraints are essential in ensuring safety.