# ST-NeRP: Spatial-Temporal Neural Representation Learning with Prior Embedding for Patient-specific Imaging Study

arXiv: https://arxiv.org/abs/2410.19283 · HTML: https://mlsanity.com/paper/2410.19283/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 34 of its 65 references that are in the map.
- First posted on arXiv 25 Oct 2024.

## Builds on (most cited first, 30 of 34)

- [DeepSDF: Learning Continuous Signed Distance Functions for Shape Representation](https://mlsanity.com/paper/1901.05103/) (2019)
- [Implicit Neural Representations with Periodic Activation Functions](https://mlsanity.com/paper/2006.09661/) (2020)
- [Fourier Features Let Networks Learn High Frequency Functions in Low Dimensional Domains](https://mlsanity.com/paper/2006.10739/) (2020)
- [D-NeRF: Neural Radiance Fields for Dynamic Scenes](https://mlsanity.com/paper/2011.13961/) (2020)
- [Swin-Unet: Unet-like Pure Transformer for Medical Image Segmentation](https://mlsanity.com/paper/2105.05537/) (2021)
- [PIFu: Pixel-Aligned Implicit Function for High-Resolution Clothed Human Digitization](https://mlsanity.com/paper/1905.05172/) (2019)
- [VoxelMorph: A Learning Framework for Deformable Medical Image Registration](https://mlsanity.com/paper/1809.05231/) (2018)
- [Unsupervised Learning of Probabilistic Diffeomorphic Registration for Images and Surfaces](https://mlsanity.com/paper/1903.03545/) (2019)
- [A Deep Learning Framework for Unsupervised Affine and Deformable Image Registration](https://mlsanity.com/paper/1809.06130/) (2018)
- [NeRP: Implicit Neural Representation Learning with Prior Embedding for Sparsely Sampled Image Reconstruction](https://mlsanity.com/paper/2108.10991/) (2021)
- [Learn2Reg: comprehensive multi-task medical image registration challenge, dataset and evaluation in the era of deep learning](https://mlsanity.com/paper/2112.04489/) (2021)
- [Quicksilver: Fast Predictive Image Registration - a Deep Learning Approach](https://mlsanity.com/paper/1703.10908/) (2017)
- [VideoINR: Learning Video Implicit Neural Representation for Continuous Space-Time Super-Resolution](https://mlsanity.com/paper/2206.04647/) (2022)
- [UniverSeg: Universal Medical Image Segmentation](https://mlsanity.com/paper/2304.06131/) (2023)
- [Weakly-Supervised Convolutional Neural Networks for Multimodal Image Registration](https://mlsanity.com/paper/1807.03361/) (2018)
- [Object-Compositional Neural Implicit Surfaces](https://mlsanity.com/paper/2207.09686/) (2022)
- [vMAP: Vectorised Object Mapping for Neural Field SLAM](https://mlsanity.com/paper/2302.01838/) (2023)
- [An Arbitrary Scale Super-Resolution Approach for 3D MR Images via Implicit Neural Representation](https://mlsanity.com/paper/2110.14476/) (2021)
- [Neural Points: Point Cloud Representation with Neural Fields for Arbitrary Upsampling](https://mlsanity.com/paper/2112.04148/) (2021)
- [GIFS: Neural Implicit Function for General Shape Representation](https://mlsanity.com/paper/2204.07126/) (2022)
- [IREM: High-Resolution Magnetic Resonance (MR) Image Reconstruction via Implicit Neural Representation](https://mlsanity.com/paper/2106.15097/) (2021)
- [Spatiotemporal implicit neural representation for unsupervised dynamic MRI reconstruction](https://mlsanity.com/paper/2301.00127/) (2022)
- [Learning Spatial-Temporal Implicit Neural Representations for Event-Guided Video Super-Resolution](https://mlsanity.com/paper/2303.13767/) (2023)
- [CNN-based Lung CT Registration with Multiple Anatomical Constraints](https://mlsanity.com/paper/2011.14372/) (2020)
- [Deep Lesion Tracker: Monitoring Lesions in 4D Longitudinal Imaging Studies](https://mlsanity.com/paper/2012.04872/) (2020)
- [One Shot Learning for Deformable Medical Image Registration and Periodic Motion Tracking](https://mlsanity.com/paper/1907.04641/) (2019)
- [Diffeomorphic Image Registration with Neural Velocity Field](https://mlsanity.com/paper/2202.12498/) (2022)
- [Learning Spatio-Temporal Model of Disease Progression with NeuralODEs from Longitudinal Volumetric Data](https://mlsanity.com/paper/2211.04234/) (2022)
- [Robust deformable image registration using cycle-consistent implicit representations](https://mlsanity.com/paper/2310.01934/) (2023)
- [Implicit Neural Representations for Generative Modeling of Living Cell Shapes](https://mlsanity.com/paper/2207.06283/) (2022)

## Built on by (most cited first, 2 of 2)

- [Artificial Intelligence-Enhanced Couinaud Segmentation for Precision Liver Cancer Therapy](https://mlsanity.com/paper/2411.02815/) (2024)
- [CPT-4DMR: Continuous sPatial-Temporal Representation for 4D-MRI Reconstruction](https://mlsanity.com/paper/2509.18427/) (2025)

## Abstract (arXiv)

During and after a course of therapy, imaging is routinely used to monitor the disease progression and assess the treatment responses. Despite of its significance, reliably capturing and predicting the spatial-temporal anatomic changes from a sequence of patient-specific image series presents a considerable challenge. Thus, the development of a computational framework becomes highly desirable for a multitude of practical applications. In this context, we propose a strategy of Spatial-Temporal Neural Representation learning with Prior embedding (ST-NeRP) for patient-specific imaging study. Our strategy involves leveraging an Implicit Neural Representation (INR) network to encode the image at the reference time point into a prior embedding. Subsequently, a spatial-temporally continuous deformation function is learned through another INR network. This network is trained using the whole patient-specific image sequence, enabling the prediction of deformation fields at various target time points. The efficacy of the ST-NeRP model is demonstrated through its application to diverse sequential image series, including 4D CT and longitudinal CT datasets within thoracic and abdominal imaging. The proposed ST-NeRP model exhibits substantial potential in enabling the monitoring of anatomical changes within a patient throughout the therapeutic journey.