# Digital Twins: McKean-Pontryagin Control for Partially Observed Physical Twins

arXiv: https://arxiv.org/abs/2510.00937 · HTML: https://mlsanity.com/paper/2510.00937/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in RL theory and sample complexity, one of 15 directions in Deep reinforcement learning methods.
- Builds on 9 of its 53 references that are in the map.
- First posted on arXiv 1 Oct 2025.

## Placed in

- Deep reinforcement learning methods / [RL theory and sample complexity](https://mlsanity.com/deeprl/rl-theory-and-sample-complexity/)

## Builds on (most cited first, 9 of 9)

- [Interacting particle solutions of Fokker-Planck equations through gradient-log-density estimation](https://mlsanity.com/paper/2006.00702/) (2020)
- [Online Planning Algorithms for POMDPs](https://mlsanity.com/paper/1401.3436/) (2014)
- [Spectral convergence of diffusion maps: improved error bounds and an alternative normalisation](https://mlsanity.com/paper/2006.02037/) (2020)
- [Manifold learning with bi-stochastic kernels](https://mlsanity.com/paper/1711.06711/) (2017)
- [Belief Space Planning: A Covariance Steering Approach](https://mlsanity.com/paper/2105.11092/) (2021)
- [Sequential Monte Carlo for Policy Optimization in Continuous POMDPs](https://mlsanity.com/paper/2505.16732/) (2025)
- [Localized Schr"odinger Bridge Sampler](https://mlsanity.com/paper/2409.07968/) (2024)
- [Ensemble Kalman-Bucy filtering for nonlinear model predictive control](https://mlsanity.com/paper/2503.12474/) (2025)
- [On a mean-field Pontryagin minimum principle for stochastic optimal control](https://mlsanity.com/paper/2506.10506/) (2025)

## Abstract (arXiv)

Optimal control for fully observed diffusion processes is well established and has led to numerous numerical implementations based on, for example, Bellman's principle, model free reinforcement learning, Pontryagin's maximum principle, and model predictive control. In contrast, much fewer algorithms are available for optimal control of partially observed processes. However, this scenario is central to the digital twin paradigm, where a physical twin is partially observed and control laws are derived based on a digital twin. In this paper, we contribute to this challenge by combining data assimilation in the form of the ensemble Kalman filter with the recently proposed McKean-Pontryagin approach to stochastic optimal control. We derive forward evolving mean-field evolution equations for states and co-states which simultaneously allow for an online assimilation of data as well as an online computation of control laws. The proposed methodology is therefore perfectly suited for real time applications of digital twins. We present numerical results for controlled Lorenz-63 and Lorenz-96 systems as well as an inverted pendulum.