# On Real-time Image Reconstruction with Neural Networks for MRI-guided Radiotherapy

arXiv: https://arxiv.org/abs/2202.05267 · HTML: https://mlsanity.com/paper/2202.05267/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 10 of its 70 references that are in the map.
- First posted on arXiv 10 Feb 2022.

## Builds on (most cited first, 10 of 10)

- [YouTube-8M: A Large-Scale Video Classification Benchmark](https://mlsanity.com/paper/1609.08675/) (2016)
- [Deep Learning Techniques for Inverse Problems in Imaging](https://mlsanity.com/paper/2005.06001/) (2020)
- [Image reconstruction by domain transform manifold learning](https://mlsanity.com/paper/1704.08841/) (2017)
- [On instabilities of deep learning in image reconstruction - Does AI come at a cost?](https://mlsanity.com/paper/1902.05300/) (2019)
- [Time-Dependent Deep Image Prior for Dynamic MRI](https://mlsanity.com/paper/1910.01684/) (2019)
- [A New Defense Against Adversarial Images: Turning a Weakness into a Strength](https://mlsanity.com/paper/1910.07629/) (2019)
- [Optimal Ratio for Data Splitting](https://mlsanity.com/paper/2202.03326/) (2022)
- [Medical Imaging and Machine Learning](https://mlsanity.com/paper/2103.01938/) (2021)
- [SANTIS: Sampling-Augmented Neural neTwork with Incoherent Structure for MR image reconstruction](https://mlsanity.com/paper/1812.03278/) (2018)
- [dAUTOMAP: decomposing AUTOMAP to achieve scalability and enhance performance](https://mlsanity.com/paper/1909.10995/) (2019)

## Built on by (most cited first, 3 of 3)

- [Uncertainty Estimation and Propagation in Accelerated MRI Reconstruction](https://mlsanity.com/paper/2308.02631/) (2023)
- [Uncertainty Estimation and Out-of-Distribution Detection for Deep Learning-Based Image Reconstruction using the Local Lipschitz](https://mlsanity.com/paper/2305.07618/) (2023)
- [Closed-loop coupling of personalised and foundation models for real-time treatment guidance with MRI](https://mlsanity.com/paper/2607.00500/) (2026)

## Abstract (arXiv)

MRI-guidance techniques that dynamically adapt radiation beams to follow tumor motion in real-time will lead to more accurate cancer treatments and reduced collateral healthy tissue damage. The gold-standard for reconstruction of undersampled MR data is compressed sensing (CS) which is computationally slow and limits the rate that images can be available for real-time adaptation. Here, we demonstrate the use of automated transform by manifold approximation (AUTOMAP), a generalized framework that maps raw MR signal to the target image domain, to rapidly reconstruct images from undersampled radial k-space data. The AUTOMAP neural network was trained to reconstruct images from a golden-angle radial acquisition, a benchmark for motion-sensitive imaging, on lung cancer patient data and generic images from ImageNet. Model training was subsequently augmented with motion-encoded k-space data derived from videos in the YouTube-8M dataset to encourage motion robust reconstruction. We find that AUTOMAP-reconstructed radial k-space has equivalent accuracy to CS but with much shorter processing times after initial fine-tuning on retrospectively acquired lung cancer patient data. Validation of motion-trained models with a virtual dynamic lung tumor phantom showed that the generalized motion properties learned from YouTube lead to improved target tracking accuracy. Our work shows that AUTOMAP can achieve real-time, accurate reconstruction of radial data. These findings imply that neural-network-based reconstruction is potentially superior to existing approaches for real-time image guidance applications.