# Real-Time Model Calibration with Deep Reinforcement Learning

arXiv: https://arxiv.org/abs/2006.04001 · HTML: https://mlsanity.com/paper/2006.04001/

## Key facts

- Cited by 12 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 7 Jun 2020.

## Built on by (most cited first, 12 of 12)

- [A Comprehensive Review of Digital Twin -- Part 1: Modeling and Twinning Enabling Technologies](https://mlsanity.com/paper/2208.14197/) (2022)
- [Uncertainty Quantification in Machine Learning for Engineering Design and Health Prognostics: A Tutorial](https://mlsanity.com/paper/2305.04933/) (2023)
- [Times Series Forecasting for Urban Building Energy Consumption Based on Graph Convolutional Network](https://mlsanity.com/paper/2105.13399/) (2021)
- [A Prescriptive Dirichlet Power Allocation Policy with Deep Reinforcement Learning](https://mlsanity.com/paper/2201.08445/) (2022)
- [Disentangling Slow and Fast Temporal Dynamics in Degradation Inference with Hierarchical Differential Models](https://mlsanity.com/paper/2509.00639/) (2025)
- [Dynamic Tsetlin Machine Accelerators for On-Chip Training at the Edge using FPGAs](https://mlsanity.com/paper/2504.19797/) (2025)
- [Prediction of the energy and exergy performance of F135 PW100 turbofan engine via deep learning](https://mlsanity.com/paper/2208.12028/) (2022)
- [Bayesian Adaptive Calibration and Optimal Design](https://mlsanity.com/paper/2405.14440/) (2024)
- [A Machine Learning Framework for Turbofan Health Estimation via Inverse Problem Formulation](https://mlsanity.com/paper/2604.08460/) (2026)
- [Optimization of geological carbon storage operations with multimodal latent dynamic model and deep reinforcement learning](https://mlsanity.com/paper/2406.04575/) (2024)
- [VertAX: a differentiable vertex model for learning epithelial tissue mechanics](https://mlsanity.com/paper/2604.06896/) (2026)
- [Trajectory Design and Budgeted Querying for Digital Twin Calibration](https://mlsanity.com/paper/2608.08631/) (2026)

## Abstract (arXiv)

The dynamic, real-time, and accurate inference of model parameters from empirical data is of great importance in many scientific and engineering disciplines that use computational models (such as a digital twin) for the analysis and prediction of complex physical processes. However, fast and accurate inference for processes with large and high dimensional datasets cannot easily be achieved with state-of-the-art methods under noisy real-world conditions. The primary reason is that the inference of model parameters with traditional techniques based on optimisation or sampling often suffers from computational and statistical challenges, resulting in a trade-off between accuracy and deployment time. In this paper, we propose a novel framework for inference of model parameters based on reinforcement learning. The contribution of the paper is twofold: 1) We reformulate the inference problem as a tracking problem with the objective of learning a policy that forces the response of the physics-based model to follow the observations; 2) We propose the constrained Lyapunov-based actor-critic (CLAC) algorithm to enable the robust and accurate inference of physics-based model parameters in real time under noisy real-world conditions. The proposed methodology is demonstrated and evaluated on two model-based diagnostics test cases utilizing two different physics-based models of turbofan engines. The performance of the methodology is compared to that of two alternative approaches: a state update method (unscented Kalman filter) and a supervised end-to-end mapping with deep neural networks. The experimental results demonstrate that the proposed methodology outperforms all other tested methods in terms of speed and robustness, with high inference accuracy.