# Temporal Differential Fields for 4D Motion Modeling via Image-to-Video Synthesis

arXiv: https://arxiv.org/abs/2505.17333 · HTML: https://mlsanity.com/paper/2505.17333/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 22 of its 35 references that are in the map.
- First posted on arXiv 22 May 2025.

## Builds on (most cited first, 22 of 22)

- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [High-Resolution Image Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2112.10752/) (2021)
- [The Unreasonable Effectiveness of Deep Features as a Perceptual Metric](https://mlsanity.com/paper/1801.03924/) (2018)
- [Video Diffusion Models](https://mlsanity.com/paper/2204.03458/) (2022)
- [Align your Latents: High-Resolution Video Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2304.08818/) (2023)
- [Towards Accurate Generative Models of Video: A New Metric & Challenges](https://mlsanity.com/paper/1812.01717/) (2018)
- [VoxelMorph: A Learning Framework for Deformable Medical Image Registration](https://mlsanity.com/paper/1809.05231/) (2018)
- [Sora: A Review on Background, Technology, Limitations, and Opportunities of Large Vision Models](https://mlsanity.com/paper/2402.17177/) (2024)
- [Learning Spatio-Temporal Representation with Pseudo-3D Residual Networks](https://mlsanity.com/paper/1711.10305/) (2017)
- [Motion-I2V: Consistent and Controllable Image-to-Video Generation with Explicit Motion Modeling](https://mlsanity.com/paper/2401.15977/) (2024)
- [LDMVFI: Video Frame Interpolation with Latent Diffusion Models](https://mlsanity.com/paper/2303.09508/) (2023)
- [Video Interpolation with Diffusion Models](https://mlsanity.com/paper/2404.01203/) (2024)
- [Diffusion Deformable Model for 4D Temporal Medical Image Generation](https://mlsanity.com/paper/2206.13295/) (2022)
- [Generating Realistic Brain MRIs via a Conditional Diffusion Probabilistic Model](https://mlsanity.com/paper/2212.08034/) (2022)
- [Cross-conditioned Diffusion Model for Medical Image to Image Translation](https://mlsanity.com/paper/2409.08500/) (2024)
- [A Spatiotemporal Volumetric Interpolation Network for 4D Dynamic Medical Image](https://mlsanity.com/paper/2002.12680/) (2020)
- [Data-Efficient Unsupervised Interpolation Without Any Intermediate Frame for 4D Medical Images](https://mlsanity.com/paper/2404.01464/) (2024)
- [4D Myocardium Reconstruction with Decoupled Motion and Shape Model](https://mlsanity.com/paper/2308.14083/) (2023)
- [PASS:Test-Time Prompting to Adapt Styles and Semantic Shapes in Medical Image Segmentation](https://mlsanity.com/paper/2410.01573/) (2024)
- [Ultrasound Image-to-Video Synthesis via Latent Dynamic Diffusion Models](https://mlsanity.com/paper/2503.14966/) (2025)
- [Learning with Explicit Shape Priors for Medical Image Segmentation](https://mlsanity.com/paper/2303.17967/) (2023)
- [SLoRD: Structural Low-Rank Descriptors for Shape Consistency in Vertebrae Segmentation](https://mlsanity.com/paper/2407.08555/) (2024)

## Built on by (most cited first, 3 of 3)

- [FIX-CLIP: Dual-Branch Hierarchical Contrastive Learning via Synthetic Captions for Better Understanding of Long Text](https://mlsanity.com/paper/2507.10095/) (2025)
- [FB-Diff: Fourier Basis-guided Diffusion for Temporal Interpolation of 4D Medical Imaging](https://mlsanity.com/paper/2507.04547/) (2025)
- [Anatomy-Guided Residual Motion Diffusion for Controllable 4D Cardiac MRI Synthesis](https://mlsanity.com/paper/2606.26764/) (2026)

## Abstract (arXiv)

Temporal modeling on regular respiration-induced motions is crucial to image-guided clinical applications. Existing methods cannot simulate temporal motions unless high-dose imaging scans including starting and ending frames exist simultaneously. However, in the preoperative data acquisition stage, the slight movement of patients may result in dynamic backgrounds between the first and last frames in a respiratory period. This additional deviation can hardly be removed by image registration, thus affecting the temporal modeling. To address that limitation, we pioneeringly simulate the regular motion process via the image-to-video (I2V) synthesis framework, which animates with the first frame to forecast future frames of a given length. Besides, to promote the temporal consistency of animated videos, we devise the Temporal Differential Diffusion Model to generate temporal differential fields, which measure the relative differential representations between adjacent frames. The prompt attention layer is devised for fine-grained differential fields, and the field augmented layer is adopted to better interact these fields with the I2V framework, promoting more accurate temporal variation of synthesized videos. Extensive results on ACDC cardiac and 4D Lung datasets reveal that our approach simulates 4D videos along the intrinsic motion trajectory, rivaling other competitive methods on perceptual similarity and temporal consistency. Codes will be available soon.