# Physics-driven Deep Learning for PET/MRI

arXiv: https://arxiv.org/abs/2206.06788 · HTML: https://mlsanity.com/paper/2206.06788/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 6 of its 40 references that are in the map.
- First posted on arXiv 11 Jun 2022.

## Builds on (most cited first, 6 of 6)

- [On instabilities of deep learning in image reconstruction - Does AI come at a cost?](https://mlsanity.com/paper/1902.05300/) (2019)
- [Direct Reconstruction of Linear Parametric Images from Dynamic PET Using Nonlocal Deep Image Prior](https://mlsanity.com/paper/2106.10359/) (2021)
- [Beyond Low Rank + Sparse: Multi-scale Low Rank Matrix Decomposition](https://mlsanity.com/paper/1507.08751/) (2015)
- [A Novel Loss Function Incorporating Imaging Acquisition Physics for PET Attenuation Map Generation using Deep Learning](https://mlsanity.com/paper/1909.01394/) (2019)
- [Attenuation Coefficient Estimation for PET/MRI With Bayesian Deep Learning pseudo-CT and Maximum Likelihood Estimation of Activity and Attenuation](https://mlsanity.com/paper/2001.03414/) (2020)
- [Utilizing the Wavelet Transform's Structure in Compressed Sensing](https://mlsanity.com/paper/2002.04150/) (2020)

## Built on by (most cited first, 2 of 2)

- [Physics-Informed Machine Learning Towards A Real-Time Spacecraft Thermal Simulator](https://mlsanity.com/paper/2407.06099/) (2024)
- [Faster Thermal Profiling of a Lunar Rover with Machine Learning Adapted Finite Difference Model](https://mlsanity.com/paper/2605.27651/) (2026)

## Abstract (arXiv)

In this paper, we review physics- and data-driven reconstruction techniques for simultaneous positron emission tomography (PET) / magnetic resonance imaging (MRI) systems, which have significant advantages for clinical imaging of cancer, neurological disorders, and heart disease. These reconstruction approaches utilize priors, either structural or statistical, together with a physics-based description of the PET system response. However, due to the nested representation of the forward problem, direct PET/MRI reconstruction is a nonlinear problem. We elucidate how a multi-faceted approach accommodates hybrid data- and physics-driven machine learning for reconstruction of 3D PET/MRI, summarizing important deep learning developments made in the last 5 years to address attenuation correction, scattering, low photon counts, and data consistency. We also describe how applications of these multi-modality approaches extend beyond PET/MRI to improving accuracy in radiation therapy planning. We conclude by discussing opportunities for extending the current state-of-the-art following the latest trends in physics- and deep learning-based computational imaging and next-generation detector hardware.