# Unsupervised inter-frame motion correction for whole-body dynamic PET using convolutional long short-term memory in a convolutional neural network

arXiv: https://arxiv.org/abs/2206.06341 · HTML: https://mlsanity.com/paper/2206.06341/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 15 of its 57 references that are in the map.
- First posted on arXiv 13 Jun 2022.

## Builds on (most cited first, 15 of 15)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [3D U-Net: Learning Dense Volumetric Segmentation from Sparse Annotation](https://mlsanity.com/paper/1606.06650/) (2016)
- [Convolutional LSTM Network: A Machine Learning Approach for Precipitation Nowcasting](https://mlsanity.com/paper/1506.04214/) (2015)
- [Attention U-Net: Learning Where to Look for the Pancreas](https://mlsanity.com/paper/1804.03999/) (2018)
- [Uformer: A General U-Shaped Transformer for Image Restoration](https://mlsanity.com/paper/2106.03106/) (2021)
- [Image Transformer](https://mlsanity.com/paper/1802.05751/) (2018)
- [VoxelMorph: A Learning Framework for Deformable Medical Image Registration](https://mlsanity.com/paper/1809.05231/) (2018)
- [Regularizing and Optimizing LSTM Language Models](https://mlsanity.com/paper/1708.02182/) (2017)
- [Quicksilver: Fast Predictive Image Registration - a Deep Learning Approach](https://mlsanity.com/paper/1703.10908/) (2017)
- [Bi-Directional ConvLSTM U-Net with Densley Connected Convolutions](https://mlsanity.com/paper/1909.00166/) (2019)
- [Combining Fully Convolutional and Recurrent Neural Networks for 3D Biomedical Image Segmentation](https://mlsanity.com/paper/1609.01006/) (2016)
- [Microscopy Cell Segmentation via Convolutional LSTM Networks](https://mlsanity.com/paper/1805.11247/) (2018)
- [Temporal HeartNet: Towards Human-Level Automatic Analysis of Fetal Cardiac Screening Video](https://mlsanity.com/paper/1707.00665/) (2017)
- [A multi-level convolutional LSTM model for the segmentation of left ventricle myocardium in infarcted porcine cine MR images](https://mlsanity.com/paper/1811.06051/) (2018)

## Built on by (most cited first, 4 of 4)

- [Fast-MC-PET: A Novel Deep Learning-aided Motion Correction and Reconstruction Framework for Accelerated PET](https://mlsanity.com/paper/2302.07135/) (2023)
- [TAI-GAN: A Temporally and Anatomically Informed Generative Adversarial Network for early-to-late frame conversion in dynamic cardiac PET inter-frame motion correction](https://mlsanity.com/paper/2402.09567/) (2024)
- [Whole-body PET image denoising for reduced acquisition time](https://mlsanity.com/paper/2303.16085/) (2023)
- [TAI-GAN: Temporally and Anatomically Informed GAN for early-to-late frame conversion in dynamic cardiac PET motion correction](https://mlsanity.com/paper/2308.12443/) (2023)

## Abstract (arXiv)

Subject motion in whole-body dynamic PET introduces inter-frame mismatch and seriously impacts parametric imaging. Traditional non-rigid registration methods are generally computationally intense and time-consuming. Deep learning approaches are promising in achieving high accuracy with fast speed, but have yet been investigated with consideration for tracer distribution changes or in the whole-body scope. In this work, we developed an unsupervised automatic deep learning-based framework to correct inter-frame body motion. The motion estimation network is a convolutional neural network with a combined convolutional long short-term memory layer, fully utilizing dynamic temporal features and spatial information. Our dataset contains 27 subjects each under a 90-min FDG whole-body dynamic PET scan. With 9-fold cross-validation, compared with both traditional and deep learning baselines, we demonstrated that the proposed network obtained superior performance in enhanced qualitative and quantitative spatial alignment between parametric Ki and Vb images and in significantly reduced parametric fitting error. We also showed the potential of the proposed motion correction method for impacting downstream analysis of the estimated parametric images, improving the ability to distinguish malignant from benign hypermetabolic regions of interest. Once trained, the motion estimation inference time of our proposed network was around 460 times faster than the conventional registration baseline, showing its potential to be easily applied in clinical settings.