# Opportunities of Hybrid Model-based Reinforcement Learning for Cell Therapy Manufacturing Process Control

arXiv: https://arxiv.org/abs/2201.03116 · HTML: https://mlsanity.com/paper/2201.03116/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 5 of its 63 references that are in the map.
- First posted on arXiv 10 Jan 2022.

## Builds on (most cited first, 5 of 5)

- [Deep Reinforcement Learning for Process Control: A Primer for Beginners](https://mlsanity.com/paper/2004.05490/) (2020)
- [Safe Chance Constrained Reinforcement Learning for Batch Process Control](https://mlsanity.com/paper/2104.11706/) (2021)
- [Constrained Model-Free Reinforcement Learning for Process Optimization](https://mlsanity.com/paper/2011.07925/) (2020)
- [Interpretable Biomanufacturing Process Risk and Sensitivity Analyses for Quality-by-Design and Stability Control](https://mlsanity.com/paper/1909.04261/) (2019)
- [Biomanufacturing Harvest Optimization with Small Data](https://mlsanity.com/paper/2101.03735/) (2021)

## Abstract (arXiv)

Driven by the key challenges of cell therapy manufacturing, including high complexity, high uncertainty, and very limited process observations, we propose a hybrid model-based reinforcement learning (RL) to efficiently guide process control. We first create a probabilistic knowledge graph (KG) hybrid model characterizing the risk- and science-based understanding of biomanufacturing process mechanisms and quantifying inherent stochasticity, e.g., batch-to-batch variation. It can capture the key features, including nonlinear reactions, nonstationary dynamics, and partially observed state. This hybrid model can leverage existing mechanistic models and facilitate learning from heterogeneous process data. A computational sampling approach is used to generate posterior samples quantifying model uncertainty. Then, we introduce hybrid model-based Bayesian RL, accounting for both inherent stochasticity and model uncertainty, to guide optimal, robust, and interpretable dynamic decision making. Cell therapy manufacturing examples are used to empirically demonstrate that the proposed framework can outperform the classical deterministic mechanistic model assisted process optimization.