# Policy Optimization in Dynamic Bayesian Network Hybrid Models of Biomanufacturing Processes

arXiv: https://arxiv.org/abs/2105.06543 · HTML: https://mlsanity.com/paper/2105.06543/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 7 of its 50 references that are in the map.
- First posted on arXiv 13 May 2021.

## Builds on (most cited first, 7 of 7)

- [Continuous control with deep reinforcement learning](https://mlsanity.com/paper/1509.02971/) (2015)
- [Stochastic First- and Zeroth-order Methods for Nonconvex Stochastic Programming](https://mlsanity.com/paper/1309.5549/) (2013)
- [On the Convergence of Stochastic Gradient Descent with Adaptive Stepsizes](https://mlsanity.com/paper/1805.08114/) (2018)
- [Non-convex Optimization for Machine Learning](https://mlsanity.com/paper/1712.07897/) (2017)
- [Reinforcement Learning for Batch Bioprocess Optimization](https://mlsanity.com/paper/1904.07292/) (2019)
- [The problem with DDPG: understanding failures in deterministic environments with sparse rewards](https://mlsanity.com/paper/1911.11679/) (2019)
- [Interpretable Biomanufacturing Process Risk and Sensitivity Analyses for Quality-by-Design and Stability Control](https://mlsanity.com/paper/1909.04261/) (2019)

## Built on by (most cited first, 4 of 4)

- [Sensitivity Analysis on Policy-Augmented Graphical Hybrid Models with Shapley Value Estimation](https://mlsanity.com/paper/2411.13396/) (2024)
- [Digital Twin Calibration with Model-Based Reinforcement Learning](https://mlsanity.com/paper/2501.02205/) (2025)
- [Variance Reduction Based Experience Replay for Policy Optimization](https://mlsanity.com/paper/2602.05379/) (2026)
- [Optimization of time-consuming experimental conditions using pseudo-experimental data guided by adaptive polynomial regression](https://mlsanity.com/paper/2607.22238/) (2026)

## Abstract (arXiv)

Biopharmaceutical manufacturing is a rapidly growing industry with impact in virtually all branches of medicines. Biomanufacturing processes require close monitoring and control, in the presence of complex bioprocess dynamics with many interdependent factors, as well as extremely limited data due to the high cost of experiments as well as the novelty of personalized bio-drugs. We develop a novel model-based reinforcement learning framework that can achieve human-level control in low-data environments. The model uses a dynamic Bayesian network to capture causal interdependencies between factors and predict how the effects of different inputs propagate through the pathways of the bioprocess mechanisms. This enables the design of process control policies that are both interpretable and robust against model risk. We present a computationally efficient, provably convergence stochastic gradient method for optimizing such policies. Validation is conducted on a realistic application with a multi-dimensional, continuous state variable.