# Med-Real2Sim: Non-Invasive Medical Digital Twins using Physics-Informed Self-Supervised Learning

arXiv: https://arxiv.org/abs/2403.00177 · HTML: https://mlsanity.com/paper/2403.00177/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- Builds on 19 of its 81 references that are in the map.
- First posted on arXiv 29 Feb 2024.

## Builds on (most cited first, 19 of 19)

- [Neural Ordinary Differential Equations](https://mlsanity.com/paper/1806.07366/) (2018)
- [Self-Supervised Learning of Pretext-Invariant Representations](https://mlsanity.com/paper/1912.01991/) (2019)
- [Characterizing possible failure modes in physics-informed neural networks](https://mlsanity.com/paper/2109.01050/) (2021)
- [Scientific Machine Learning through Physics-Informed Neural Networks: Where we are and What's next](https://mlsanity.com/paper/2201.05624/) (2022)
- [Deep Convolutional Neural Network for Inverse Problems in Imaging](https://mlsanity.com/paper/1611.03679/) (2016)
- [Augmented Neural ODEs](https://mlsanity.com/paper/1904.01681/) (2019)
- [DeblurGAN: Blind Motion Deblurring Using Conditional Adversarial Networks](https://mlsanity.com/paper/1711.07064/) (2017)
- [B-PINNs: Bayesian Physics-Informed Neural Networks for Forward and Inverse PDE Problems with Noisy Data](https://mlsanity.com/paper/2003.06097/) (2020)
- [Deep Learning Techniques for Inverse Problems in Imaging](https://mlsanity.com/paper/2005.06001/) (2020)
- [Deep Learning for Segmentation using an Open Large-Scale Dataset in 2D Echocardiography](https://mlsanity.com/paper/1908.06948/) (2019)
- [hp-VPINNs: Variational Physics-Informed Neural Networks With Domain Decomposition](https://mlsanity.com/paper/2003.05385/) (2020)
- [Image reconstruction by domain transform manifold learning](https://mlsanity.com/paper/1704.08841/) (2017)
- [Informed Machine Learning -- A Taxonomy and Survey of Integrating Knowledge into Learning Systems](https://mlsanity.com/paper/1903.12394/) (2019)
- [A Deep Learning Approach to Structured Signal Recovery](https://mlsanity.com/paper/1508.04065/) (2015)
- [SPINN: Sparse, Physics-based, and partially Interpretable Neural Networks for PDEs](https://mlsanity.com/paper/2102.13037/) (2021)
- [Physics-informed neural networks for myocardial perfusion MRI quantification](https://mlsanity.com/paper/2011.12844/) (2020)
- [Global Guarantees for Blind Demodulation with Generative Priors](https://mlsanity.com/paper/1905.12576/) (2019)
- [Learning atrial fiber orientations and conductivity tensors from intracardiac maps using physics-informed neural networks](https://mlsanity.com/paper/2102.10863/) (2021)
- [Deep learning for inverse problems with unknown operator](https://mlsanity.com/paper/2108.02744/) (2021)

## Built on by (most cited first, 8 of 8)

- [Guardian-regularized Safe Offline Reinforcement Learning for Smart Weaning of Mechanical Circulatory Devices](https://mlsanity.com/paper/2511.06111/) (2025)
- [Revolutionizing Clinical Trials: A Manifesto for AI-Driven Transformation](https://mlsanity.com/paper/2506.09102/) (2025)
- [Anatomy-DT: A Cross-Diffusion Digital Twin for Anatomical Evolution](https://mlsanity.com/paper/2509.25280/) (2025)
- [SoC-DT: Standard-of-Care Aligned Digital Twins for Patient-Specific Tumor Dynamics](https://mlsanity.com/paper/2510.03287/) (2025)
- [Interaction-Limited Safe Continuous-Time RL for Dynamical Medical Treatment](https://mlsanity.com/paper/2606.01051/) (2026)
- [DT2: Decision-Targeted Digital Twins](https://mlsanity.com/paper/2606.25923/) (2026)
- [Cardiovascular Digital Twins from Physics Based to Data Driven Approaches](https://mlsanity.com/paper/2608.02135/) (2026)
- [Impact of Multiple Non-Invasive Biosignals on Cardiovascular Biomarker Estimation via Simulation-Based Inference](https://mlsanity.com/paper/2609.11969/) (2026)

## Abstract (arXiv)

A digital twin is a virtual replica of a real-world physical phenomena that uses mathematical modeling to characterize and simulate its defining features. By constructing digital twins for disease processes, we can perform in-silico simulations that mimic patients' health conditions and counterfactual outcomes under hypothetical interventions in a virtual setting. This eliminates the need for invasive procedures or uncertain treatment decisions. In this paper, we propose a method to identify digital twin model parameters using only noninvasive patient health data. We approach the digital twin modeling as a composite inverse problem, and observe that its structure resembles pretraining and finetuning in self-supervised learning (SSL). Leveraging this, we introduce a physics-informed SSL algorithm that initially pretrains a neural network on the pretext task of learning a differentiable simulator of a physiological process. Subsequently, the model is trained to reconstruct physiological measurements from noninvasive modalities while being constrained by the physical equations learned in pretraining. We apply our method to identify digital twins of cardiac hemodynamics using noninvasive echocardiogram videos, and demonstrate its utility in unsupervised disease detection and in-silico clinical trials.